There are two types of memory errors and both require a different solution.
The most common error: "java.lang.OutOfMemoryError". This problem is relatively easy to find and often the stacktrace even mentions the cause. There is one or many processes that consume a large amount of data, so large that it can not add more to the Java Heap (memory).
Most often the microflow is retrieving/evaluating to much data and is running out of memory.
The error you are having is the "GC overhead limit exceeded". Unlike the out of memory error, this does not necessarily indicate that you have to many records in your cache.
Unfortunately this is the most difficult problem to trace or solve. You will need to analyze the memory usage graphs and identify what is causing the problem.
The GC limit is a statistic that Java keeps track of. You are getting this error if Java is spending about 98% of the time actively garbage collecting. This can have many different causes, I'm just going to give a few simple examples here. But as I said you'll have to compare your memory usage statistics with the processes that run at a specific time to draw the right conclusion and act accordingly.
Potential causes
You are executing an extremely high volume of microflows that use many objects that have to live in memory for a short period of time. If the nr of microflows and the nr of objects is high enough, it could cause memory to grow faster than it can be collected.
- Solution 1: Increase the interval so the garbage collector has time to garbage collect between microflows
- Solution 2: Improve memory usage and use batches / improve efficiency of the microflow
There is or are several microflows that execute for a long time, and consume most of the available memory. This leaves only a small percentage of the Java Heap for all other processes, such as interaction with the client, Mx background jobs, etc.
- Solution: Improve memory usage and use batches / improve efficiency of the microflow
- Additional action: Increase the available memory for the server
There are several users active on the system all working with semi-large data sets. The nr of actions, and thus the nr of objects in memory is so high and changing so frequently that Java can't keep up with garbage collection before risking to run out of memory.
- Solution: Increase the available memory for the server. This will give Java more time to garbage collect before the Java Heap is full again.
The memory usage of the application is slowly increasing over time. In the memory graph it shows that the garbage collection doesn't completely free up all used space. Which indicates a memory leak.
- Solution: Search through the records that are in the object cache and several random points in time. Identify the objects and session that is keeping the objects. Identify the potential microflows, etc.. and try and solve the memory leak.
One last thing about these errors and their stacktraces, the processes mentioned in these stacktraces are usually not the culprit of these problems. Especially the garbage collection problem, the error happens while initiating a new garbage collection action. GC never starts while running or finishing an activity. Since a microflow can't start a GC action, the cause of your problem (the microflow) will never show in the stacktrace.
The same can also apply to the out of memory error, it can also be that last datagrid with just 5 records in it that is opened by the user. If those 5 objects no longer fit in memory that is what you will see in the stacktrace, not the microflow that might be retrieving a million objects.
See also this Oracle page on the difference between the two errors: https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/memleaks002.html (FYI, i
Could it be that the import doesn't use all the resources, but when trying to clean up, it actually runs out of memory? You could try to increase the heapsize using, for example, "-Xmx1024m"