Tracing - loop span has a significant overhead not caused by iterations

0
Hi all!So, I have noticed a few having an impact on the performance, but I am not sure how to interpret this specific issue. As stated in the title: loop span has a significant overhead not caused by iterations. And then I looked in other places, seems to be the case there, too. So, the question is: How to interpret this part of a loop span not attributable to loop iterations? Is it normal? How to optimize that, as it is not insignificant?Thanks in advance!
asked
3 answers
0

Hi,



What you are seeing is generally normal when looking at loop spans in Mendix tracing. The time shown for the loop span is not only the sum of the iterations. It also includes the overhead of the loop construct itself and some runtime activities that are not attributed to the individual iteration spans.

For example, the loop span may include:

  • Retrieving or preparing the list that the loop iterates over
  • Context switching and runtime bookkeeping between iterations
  • Logging and tracing instrumentation itself
  • Transaction management if the loop runs inside a larger microflow transaction

Because of that, the loop span can appear noticeably larger than the combined time of the iterations.

Another important point is that tracing aggregates some operations at the parent span level. If certain activities happen between iterations or before the first iteration starts, they may be recorded only in the loop span and not inside a specific iteration span.

In terms of optimization, the loop overhead itself is usually not the main bottleneck. If the loop becomes expensive, the typical areas to review are:

  • Repeated database queries inside the loop
  • Commits happening per iteration
  • Expensive calculations or microflow calls within each iteration
  • Large object lists being processed in memory

A good approach is to enable deeper tracing on the microflows executed inside the loop and focus on the operations performed during each iteration rather than the loop construct itself.

So in short, seeing some additional time in the loop span that is not directly attributed to iterations is expected behavior in Mendix tracing and does not necessarily indicate a performance issue by itself.


answered
0

Have lovely day Nenad, :)


The part of the loop span that is not attributed to the iterations typically represents the loop overhead handled by the Mendix Runtime. This includes tasks such as preparing the list for iteration, managing loop context variables, evaluating the loop condition, and internal runtime bookkeeping between iterations. Monitoring and tracing instrumentation can also contribute to this overhead. Because of this, it is common to see some time in the loop span that is not directly linked to the individual iteration spans.


In most cases, this does not indicate a specific problem with the loop itself. The profiler shows the total execution time of the loop container, while only part of that time can be mapped to the actions executed inside each iteration.


Instead of focusing on the loop container time, the best approach is to analyze what happens inside the loop and how many objects are processed. Many performance issues occur when loops run over large lists or when expensive operations are executed inside each iteration.


One important best practice is to avoid database operations inside loops. For example, repeated Retrieve, Commit, or Web Service calls inside a loop can significantly slow down performance. If possible, retrieve the required data before the loop starts and process it in memory.


Another good practice is to move logic that does not change outside the loop. If a value, calculation, or retrieve is the same for every iteration, it should be executed once before the loop rather than repeatedly during each iteration.


You should also check the size of the list being processed. Sometimes loops run over more objects than necessary. Applying filters in the retrieve step or reducing the dataset before the loop can significantly improve performance.


Additionally, try to avoid nested loops where possible. Nested loops increase complexity and execution time exponentially. In many cases, using associations, list operations, or pre-processed data structures can eliminate the need for nested loops.


For more details on Mendix performance and loop optimization, you can refer to the official documentation:

https://docs.mendix.com/refguide/performance-best-practices/


If this resolves your issue, please mark the answer as accepted so it can help others in the community as well.


answered
0

Hi Nenad Todorović,


In Mendix, a loop’s total time includes hidden work outside the actual iteration body, such as:

  • List retrieval + materialization (XPath, associations, security checks)
  • Lazy loads / N+1 queries when accessing associations inside the loop
  • Commits, events, and transaction overhead happening before/after iterations
  • DB lock waits / connection waits
  • GC or runtime pauses


So yes it’s normal, but it’s often a sign something can be optimized.


How to optimize


1. Retrieve smart

  • Filter in the DB, not in the loop
  • Prefetch associations if used inside the loop (avoid N+1)

2. Don’t commit inside the loop

  • Change all objects → commit once after
  • Or batch commit (e.g., every 200)

3. Avoid heavy expressions inside the loop

  • Move constants, calculated attributes, and list operations outside

4. Watch entity access & events

  • Disable events or access rules for internal flows if safe

5. Consider chunking

Process large lists in parts to reduce memory + GC pressure.


Note!!!!:If the loop span is large but iterations look cheap, the cost is data retrieval, lazy loading, or commit overhead, not the loop itself.


I hope this help's



answered