Will clearing a list after commiting the list improve memory usage?

4
Hi everyone. Imagine a microflow which contains two consecutive submicroflow calls (1 and 2). In the first submicroflow, I create a list. Then I loop over 25000 objects, change something and add each object to the list. Then I commit the list outside the loop. This is a standard procedure to avoid committing 25000 objects. Question 1: What happens when submicroflow 1 is finished? Will the list be cleared? Will it still use a lot of memory when the flow starts submicroflow 2? Will clearing the list at the end of submicroflow 1 (after commiting of course) improve my memory usage in any way? Question 2: I retrieve a list of 25000 objects from the database and perform some action on its elements. I commit my changes and move on. I no longer need the list, but my microflow isn't finished yet. Will clearing the list improve my memory usage in any way? Thank you. Koen
asked
2 answers
5

Answer 1: at this page, you can read:

"It is important to note that the previous definition used the word 'action’. The garbage collector will not run after each microflow, for example a sub-microflow will not initiate the garbage collector. Any action that requires a transaction, such as a click on a microlfow button, a click on the save button, scheduled events, etc. are all actions that can initiate the garbage collector."

Since your microflows are run in the same transaction, nothing happens. The list is not cleared, it still uses memory, because it is still in use.

Answer 2: No, because your transaction is not finished, so all these changes have not been committed to the database, and are still in memory.

What you (probably) should do is implement batching behavior, as shown here. Note: although this is from the 4.0 reference guide, it is still applicable to Mendix 5. Mendix optimizes this batching pattern to reduce the memory required, I believe by doing intermediate commits.

[edit] In response to Stephan:

I created a simple microflow (see here) which updates a string attribute of a list of objects. Then, I clear the list. While debugging, I set the log levels to trace to see what is happening. I see the following in the log:

On the commit action, SQL UPDATE queries are executed. After the end action in the microflow, the following information can be seen in the log:

  • Get all changed objects through an SQL call (1)
  • 100 the message:

Attempted to add id [MendixIdentifier:: id=3659174697238529 objectType=MyFirstModule.SomeEntity entityID=13] to client rootset but it was already present.

From this, I conclude that:

  • Clearing a list does not help (a lot), since at the end of the microflow call, all changed objects are retrieved from the database. However, only the changed fields are retrieved, so for objects with many attributes with a limited change set, this may be beneficial.
  • Clearing a list does not help, since the objects are still present in the client rootset (i.e. they still live on in Mendix in some form, instead of merely living in database memory).

(1) SQL call:

SQL: SELECT "myfirstmodule$someentity"."id", "myfirstmodule$someentity"."stringtobeset" FROM "myfirstmodule$someentity" WHERE "myfirstmodule$someentity"."id" IN (3659174697238529, 3659174697238530, 3659174697238531, 3659174697238532, 3659174697238533, 3659174697238534, 3659174697238535, 3659174697238536, 3659174697238537, 3659174697238538, 3659174697238539, 3659174697238540, 3659174697238541, 3659174697238542, 3659174697238543, 3659174697238544, 3659174697238545, 3659174697238546, 3659174697238547, 3659174697238548, 3659174697238549, 3659174697238550, 3659174697238551, 3659174697238552, 3659174697238553, 3659174697238554, 3659174697238555, 3659174697238556, 3659174697238557, 3659174697238558, 3659174697238559, 3659174697238560, 3659174697238561, 3659174697238562, 3659174697238563, 3659174697238564, 3659174697238565, 3659174697238566, 3659174697238567, 3659174697238568, 3659174697238569, 3659174697238570, 3659174697238571, 3659174697238572, 3659174697238573, 3659174697238574, 3659174697238575, 3659174697238576, 3659174697238577, 3659174697238578, 3659174697238579, 3659174697238580, 3659174697238581, 3659174697238582, 3659174697238583, 3659174697238584, 3659174697238585, 3659174697238586, 3659174697238587, 3659174697238588, 3659174697238589, 3659174697238590, 3659174697238591, 3659174697238592, 3659174697238593, 3659174697238594, 3659174697238595, 3659174697238596, 3659174697238597, 3659174697238598, 3659174697238599, 3659174697238600, 3659174697238601, 3659174697238602, 3659174697238603, 3659174697238604, 3659174697238605, 3659174697238606, 3659174697238607, 3659174697238608, 3659174697238609, 3659174697238610, 3659174697238611, 3659174697238612, 3659174697238613, 3659174697238614, 3659174697238615, 3659174697238616, 3659174697238617, 3659174697238618, 3659174697238619, 3659174697238620, 3659174697238621, 3659174697238622, 3659174697238623, 3659174697238624, 3659174697238625, 3659174697238626, 3659174697238627, 3659174697238628)

answered
0

In response to Rom van Arendonk's answer:

Answer 1) Agree

Answer 2) Disagree

I believe there is a performance gain when you clear lists, since memory used by Mendix != database memory. I wrote on my blog the following:

The clear list action empties the objects from the targeted list, consequently the objects in the list are unavailable further on in the microflow. The load on memory can benefit greatly from the clear list action, specially in large memory consuming microflows. However clearing a list also takes up resources from the server. To understand the dynamics of memory consumption we need to look at two memory management systems on top of each other: “the Mendix Platform internal memory, which stores all non-persistent objects and all changed objects. And the regular Java Garbage collector which cleans up all objects as soon as the Platform is done with them. The built in Mendix garbage collector will not execute until the end of the microflow. That means that when you change an object, those changes will sit somewhere in memory as well. For the Mendix garbage collector it makes no difference since the Mx Garbage Collector will not be executed until the end of the microflow. There is also the Java Garbage Collector. This process runs whenever needed and cleans up all objects that are no longer referenced from memory. If you remove items from the list it will allow the JGC to collect those objects and free memory more…(Mx). Make sure that you commit the changes first, because that will ensure that these objects are added to the database transaction and kept in database memory till they are really stored in the database (end of transaction/microflow is reached).

I refer to the answer provided by Jasper van der Hoek in this topic

@Rom could you please elaborate why you think this wouldn't work?

With kind regards, Stephan

answered