Java heap space

0
Hi everyone, I’m working on a app where users upload an Excel file which has employee data, the system validates each line (over 100k lines, 25 columns), and if everything is valid, it sends an email and then converts the data to CSV and then delete the EmployeeData. I have two main entities: ExcelFile and EmployeeData (1 file has many employee records). Validation runs in a task queue (threads: 2). Validation, csv conversion and deletion happen in a batch process (3000). Env size M21-STANDARD. Will it help us changing the size of env.? I have seen similar issues on the Mx community. And i know that i'm trying to process too many records at once. The problem: When I upload only one large or small file (e.g., 200k lines), everything works fine.When i upload multiple small files it works. But when I upload multiple large files one after another, the app sometimes stops mid-process, usually after the validation step when tries to send mail.  However i disable this SUB in this case it stops in the csv conversion. The error:  java.lang.OutOfMemoryError: Java heap space ERROR - TaskQueue: Failed to retrieve next task from task queue 'System.MendixWorkflows-DefaultTaskExecution': An exception has occurred...   Did anybody face same/similar issue? Can someone help me to solve this problem? Any suggestion is welcome. Thanks in advance.
asked
4 answers
2

The Excel import module is using a lot of memory. Consider using Streaming excel.

 

 

answered
0

Some benchmarking of the individual processes (e.g. using visualvm) can help identifying the bottleneck.

answered
0

I think it's going to be easier to use this datagrid 2 excel export https://docs.mendix.com/appstore/modules/data-grid-2/#export-to-excel

answered
0

Hi selcuk,

    Increase Environment Size

  • M21-STANDARD has ~2–4 GB of heap depending on config.

  • Moving to a larger environment (e.g., M24 or M32) gives you more heap space.

  •  Yes, upgrading can help—but you should also optimize to avoid scaling cost unnecessarily.

answered