uploading huge csv file

[Magickiller9]

Hi guys,

I'm trying to upload 3 huge csv file of 400+MB, 1MB and 400KB respectively in a zip folder using Java.io.* to upload and unzip files.

But i'm hitting an issue that when i unzip the file. I'm only getting 35.1MB for the largest file instead of the full 400MB.

Tried increasing buffer size etc but not working. Not sure where did i miss out.

 

[davidktw]

Hi guys,

I'm trying to upload 3 huge csv file of 400+MB, 1MB and 400KB respectively in a zip folder using Java.io.* to upload and unzip files.

But i'm hitting an issue that when i unzip the file. I'm only getting 35.1MB for the largest file instead of the full 400MB.

Tried increasing buffer size etc but not working. Not sure where did i miss out.

Buffer size has nothing to do with your file size, as well as getting data read/write correctly across. The most it will impact on is performance.

Place debugging codes to find out how much data you have read in and how much you have written out.

Maybe you want to post fragments of your codes to demonstrate how you are reading in the file and how you are sending out into the network. Provide the client and server side of the codes, if you are developing both ends.

 

[u0206397]

Before uploading, write your ZIP file to a local file. Check if it can be opened and decompressed properly using a third party ZIP tool like Winzip or pkzip.

If you are uploading files to a web server, check your web server file upload size limit. Depending on your server side implementation, this could be set at different places. Example for PHP is in /etc/php.ini file. For Java servlet/JSP, it could be somewhere in a WEB-INF/web.xml in your web app. You need to check where the file size threshold is set.

If you are unlucky, it could also be hidden in some network settings, where the web proxy, antivirus or firewall is configured to truncate your HTTP request upon exceeding certain length. In most normal circumstances, it shouldn't, and should just either block your request outright or let your connection pass in entirety.