uploading large file approx-100000 rows. file size - 25 MB

0
When uploading large excel file then getting error.   error occured:- if rows>20000 our need is to uploading large file approx-100000 rows. file size - 25 MB
asked
4 answers
0

Hi Gopal, you can change the uploading file settings from uploading widget to set it whichever size you want. But It is not efficient to use that large file for importer which may get time to upload it. It may not completed in a short time that may bring other errors.

answered
0

Hi Gopal,

If it is related to OutOfMemory error (heap space), then you can try increasing the size of your container and allocate more memory to JVM.

answered
0

Hi Gopal,

Uploading large files, such as a 25 MB Excel file with approximately 100,000 rows, can be challenging due to resource limitations and performance considerations in web applications.

  1. Size Limit: Mendix has default limits on file upload size. You need to increase the maximum file size allowed in Mendix settings to handle large files.

  2. Break into Chunks: Split the large file into smaller pieces (chunks) on the client-side. Send these smaller chunks to the server instead of the entire file at once.

  3. Asynchronous Processing: Process the chunks asynchronously to avoid exceeding the maximum execution time of a single request.

  4. Memory Management: Optimize code to use memory efficiently. Avoid loading the entire file into memory all at once.

  5. Database Optimization: Optimize database interactions for efficiency when handling large amounts of data.

  6. Error Handling: Implement proper error handling to deal with any issues that may occur during file upload and processing.

  7. Consider Alternatives: For extremely large files, explore alternatives like direct database imports or using cloud storage services.

answered
0

Dear Gopal if possible I would import the excel file as CSV, I have imported 10000+ rows with a csv importer without any problem. 

answered