Mendix Cloud v4 Critical Alert: Max # of Threads in use. False Positiv error?

2
Hi folks,  we have got an application running in the Mendix Cloud v4. Today I got a critical alert-mail from the cloud. After looking into the error log, I've found following error:  2018-04-03T08:28:25.18+0000 [APP/PROC/WEB/0]    CRITICAL - M2EE: Low on resources, maximum number of threads are in use (254) I've also checked the monitoring of the node in that timeframe and it was the jetty threadpool which has reached the maximum. But the actual number of threads wasn't even near the maximum. (see image below)   So I am not a server/infrastructure expert. My question is: Is this a genuine alert or just a false positive alert?  Is there something I can do to prevent this error?
asked
2 answers
2

Hi Alexander

This is the threadpool for the buidin jetty. More information on the Mendix monitoring page:
Threadpool for Handling External Requests

The application server thread pool graph shows the number of concurrent requests that are being handled bij the Mendix Runtime, but only when they’re initiated by a remote API, like the way the normal web-based client communicates, or by calling web services. Because creating a new thread that can concurrently process a request is an expensive operation, there’s a pool of threads being held that can quickly start processing new incoming requests. This pool automatically grows and shrinks according to the number of requests that are flowing through the application.

 

When i look at the graph in your post the 'threadpool size' did reach the 'max thread size'. However, what is not clear from the documention or the graph is how the 'threadpool size' is determined. You would expect the idle + active threads together will determine the  'threadpool size'. The idle threads are not visible in the graph.

In a runtime environment (on-premise) it is possible to get more information from the m2ee. There the idle_threads are also visible. Also see:

https://docs.mendix.com/refguide/monitoring-mendix-runtime#5-server-statistics

So, possibly there were a lot of idle threads. Something triggered from client building up a lot of threads.  By default the threads do have a idle timeout.  

I would advice to ask Mendix support.

More detailed information about the QueuedThreadPool:

http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/util/thread/QueuedThreadPool.html

 

answered
1

Hi Alexander,

Do you know if there are many simultaneous connections that remain open to the Mendix application (e.g. synchronously running microflows, database connections, API call)?

Each connection uses a thread from the jetty thread pool.

answered