Many uploaded files are camera shots from iOS devices. I would estimate these to be under 4 MB, on average. Multiple files are being uploaded together. These files consist of documents that prove identity and employment.
I will reply with another comment with hard numbers for those that have uploaded successfully. Thanks for your inquiry.
That's a good suggestion, Abhishek. I'll post an update with results later. Thanks!
Unfortunately, we have not been able to find a solution for this issue. It has happened 626 times in production over the past month. I filed a Mendix ticket, but i have gotten no resolution there, either. The most recent error log is attached.
2023-05-11T15:57:33.199568 [APP/PROC/WEB/1] ERROR - Connector: Uploading file failed: Unable to extract content parameters [User 'Anonymous_b6d33ea0-29ec-450c-ab94-8c67ebaa6d01' with session id 'fa901ebc-XXXX-XXXX-XXXX-XXXXXXXXaafb' and roles 'Anonymous']
2023-05-11T15:57:33.199582 [APP/PROC/WEB/1] ERROR - Connector: org.eclipse.jetty.io.RuntimeIOException: Unable to extract content parameters
2023-05-11T15:57:33.199586 [APP/PROC/WEB/1] at org.eclipse.jetty.server.Request.extractContentParameters(Request.java:559)
2023-05-11T15:57:33.199596 [APP/PROC/WEB/1] at org.eclipse.jetty.server.Request.getParameters(Request.java:460)
2023-05-11T15:57:33.199661 [APP/PROC/WEB/1] at org.eclipse.jetty.server.Request.getParameter(Request.java:1046)
2023-05-11T15:57:33.199752 [APP/PROC/WEB/1] at com.mendix.m2ee.appcontainer.server.request.HttpMxRuntimeRequest.getParameter(HttpMxRuntimeRequest.scala:32)
2023-05-11T15:57:33.199766 [APP/PROC/WEB/1] at com.mendix.webui.requesthandling.FileUploadHelper.readFilePartsFromRequest(FileUploadHelper.scala:163)
2023-05-11T15:57:33.199821 [APP/PROC/WEB/1] at com.mendix.webui.requesthandling.FileUploadHelper.checkFileForUpload(FileUploadHelper.scala:95)
2023-05-11T15:57:33.199917 [APP/PROC/WEB/1] at com.mendix.webui.requesthandling.FileUploadHelper.doUpload(FileUploadHelper.scala:123)
2023-05-11T15:57:33.199987 [APP/PROC/WEB/1] at com.mendix.webui.requesthandling.FileUploadHelper.handleUpload(FileUploadHelper.scala:43)
2023-05-11T15:57:33.200042 [APP/PROC/WEB/1] at com.mendix.webui.requesthandling.FileRequestHandler.handleRequest(FileRequestHandler.scala:44)
2023-05-11T15:57:33.200099 [APP/PROC/WEB/1] at com.mendix.webui.requesthandling.FileRequestHandler.processRequest(FileRequestHandler.scala:32)
2023-05-11T15:57:33.200161 [APP/PROC/WEB/1] at com.mendix.externalinterface.connector.RequestHandler.doProcessRequest(RequestHandler.java:37)
2023-05-11T15:57:33.200226 [APP/PROC/WEB/1] at com.mendix.external.connector.MxRuntimeConnector.$anonfun$processRequest$1(MxRuntimeConnector.scala:54)
2023-05-11T15:57:33.200330 [APP/PROC/WEB/1] at com.mendix.external.connector.MxRuntimeConnector.$anonfun$processRequest$1$adapted(MxRuntimeConnector.scala:54)
2023-05-11T15:57:33.200379 [APP/PROC/WEB/1] at com.mendix.util.classloading.Runner$.withContextClassLoader(Runner.scala:20)
2023-05-11T15:57:33.200442 [APP/PROC/WEB/1] at com.mendix.external.connector.MxRuntimeConnector.processRequest(MxRuntimeConnector.scala:54)
2023-05-11T15:57:33.200510 [APP/PROC/WEB/1] at com.mendix.basis.impl.MxRuntimeImpl.processRequest(MxRuntimeImpl.scala:221)
2023-05-11T15:57:33.200553 [APP/PROC/WEB/1] at com.mendix.m2ee.appcontainer.server.handler.RuntimeServlet.service(RuntimeServlet.scala:40)
2023-05-11T15:57:33.200629 [APP/PROC/WEB/1] at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
2023-05-11T15:57:33.200730 [APP/PROC/WEB/1] at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:764)
2023-05-11T15:57:33.200743 [APP/PROC/WEB/1] at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1665)
2023-05-11T15:57:33.200793 [APP/PROC/WEB/1] at org.eclipse.jetty.websocket.servlet.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:170)
2023-05-11T15:57:33.200836 [APP/PROC/WEB/1] at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
2023-05-11T15:57:33.200869 [APP/PROC/WEB/1] at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
2023-05-11T15:57:33.200957 [APP/PROC/WEB/1] at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527)
2023-05-11T15:57:33.200994 [APP/PROC/WEB/1] at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)
2023-05-11T15:57:33.201040 [APP/PROC/WEB/1] at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1571)
2023-05-11T15:57:33.201088 [APP/PROC/WEB/1] at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)
2023-05-11T15:57:33.201153 [APP/PROC/WEB/1] at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1378)
2023-05-11T15:57:33.201183 [APP/PROC/WEB/1] at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176)
2023-05-11T15:57:33.201250 [APP/PROC/WEB/1] at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484)
2023-05-11T15:57:33.201281 [APP/PROC/WEB/1] at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1544)
2023-05-11T15:57:33.201356 [APP/PROC/WEB/1] at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174)
2023-05-11T15:57:33.201390 [APP/PROC/WEB/1] at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1300)
2023-05-11T15:57:33.201446 [APP/PROC/WEB/1] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129)
2023-05-11T15:57:33.201513 [APP/PROC/WEB/1] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
2023-05-11T15:57:33.201572 [APP/PROC/WEB/1] at org.eclipse.jetty.server.Server.handle(Server.java:562)
2023-05-11T15:57:33.201633 [APP/PROC/WEB/1] at org.eclipse.jetty.server.HttpChannel.lambda$handle$0(HttpChannel.java:505)
2023-05-11T15:57:33.201722 [APP/PROC/WEB/1] at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:762)
2023-05-11T15:57:33.201780 [APP/PROC/WEB/1] at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:497)
2023-05-11T15:57:33.201835 [APP/PROC/WEB/1] at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:282)
2023-05-11T15:57:33.201936 [APP/PROC/WEB/1] at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:319)
2023-05-11T15:57:33.201951 [APP/PROC/WEB/1] at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
2023-05-11T15:57:33.202025 [APP/PROC/WEB/1] at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
2023-05-11T15:57:33.202065 [APP/PROC/WEB/1] at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:412)
2023-05-11T15:57:33.202134 [APP/PROC/WEB/1] at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:381)
2023-05-11T15:57:33.202188 [APP/PROC/WEB/1] at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:268)
2023-05-11T15:57:33.202263 [APP/PROC/WEB/1] at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.lambda$new$0(AdaptiveExecutionStrategy.java:138)
2023-05-11T15:57:33.202281 [APP/PROC/WEB/1] at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:407)
2023-05-11T15:57:33.202371 [APP/PROC/WEB/1] at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:894)
2023-05-11T15:57:33.202421 [APP/PROC/WEB/1] at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1038)
2023-05-11T15:57:33.202483 [APP/PROC/WEB/1] at java.base/java.lang.Thread.run(Unknown Source)
2023-05-11T15:57:33.202604 [APP/PROC/WEB/1] Caused by: java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout expired: 30000/30000 ms
2023-05-11T15:57:33.202645 [APP/PROC/WEB/1] at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:307)
2023-05-11T15:57:33.202742 [APP/PROC/WEB/1] at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:272)
2023-05-11T15:57:33.202760 [APP/PROC/WEB/1] at java.base/java.io.BufferedInputStream.read1(Unknown Source)
2023-05-11T15:57:33.202835 [APP/PROC/WEB/1] at java.base/java.io.BufferedInputStream.read(Unknown Source)
2023-05-11T15:57:33.202888 [APP/PROC/WEB/1] at java.base/java.io.FilterInputStream.read(Unknown Source)
2023-05-11T15:57:33.202919 [APP/PROC/WEB/1] at org.eclipse.jetty.server.MultiPartFormInputStream.parse(MultiPartFormInputStream.java:580)
2023-05-11T15:57:33.203246 [APP/PROC/WEB/1] at org.eclipse.jetty.server.MultiPartFormInputStream.getParts(MultiPartFormInputStream.java:474)
2023-05-11T15:57:33.203264 [APP/PROC/WEB/1] at org.eclipse.jetty.server.MultiParts$MultiPartsHttpParser.getParts(MultiParts.java:85)
2023-05-11T15:57:33.203306 [APP/PROC/WEB/1] at org.eclipse.jetty.server.Request.getParts(Request.java:2298)
2023-05-11T15:57:33.203332 [APP/PROC/WEB/1] at org.eclipse.jetty.server.Request.extractContentParameters(Request.java:552)
2023-05-11T15:57:33.203401 [APP/PROC/WEB/1] at org.eclipse.jetty.server.Request.getParameters(Request.java:460)
2023-05-11T15:57:33.203413 [APP/PROC/WEB/1] at org.eclipse.jetty.server.Request.getParameter(Request.java:1046)
2023-05-11T15:57:33.203419 [APP/PROC/WEB/1] at com.mendix.m2ee.appcontainer.server.request.HttpMxRuntimeRequest.getParameter(HttpMxRuntimeRequest.scala:32)
2023-05-11T15:57:33.203456 [APP/PROC/WEB/1] at com.mendix.webui.requesthandling.FileUploadHelper.readFilePartsFromRequest(FileUploadHelper.scala:163)
2023-05-11T15:57:33.203461 [APP/PROC/WEB/1] at com.mendix.webui.requesthandling.FileUploadHelper.checkFileForUpload(FileUploadHelper.scala:95)
2023-05-11T15:57:33.203506 [APP/PROC/WEB/1] at com.mendix.webui.requesthandling.FileUploadHelper.doUpload(FileUploadHelper.scala:123)
2023-05-11T15:57:33.203519 [APP/PROC/WEB/1] at com.mendix.webui.requesthandling.FileUploadHelper.handleUpload(FileUploadHelper.scala:43)
2023-05-11T15:57:33.203532 [APP/PROC/WEB/1] at com.mendix.webui.requesthandling.FileRequestHandler.handleRequest(FileRequestHandler.scala:44)
2023-05-11T15:57:33.203544 [APP/PROC/WEB/1] at com.mendix.webui.requesthandling.FileRequestHandler.processRequest(FileRequestHandler.scala:32)
2023-05-11T15:57:33.203623 [APP/PROC/WEB/1] at com.mendix.externalinterface.connector.RequestHandler.doProcessRequest(RequestHandler.java:37)
2023-05-11T15:57:33.203662 [APP/PROC/WEB/1] at com.mendix.external.connector.MxRuntimeConnector.$anonfun$processRequest$1(MxRuntimeConnector.scala:54)
2023-05-11T15:57:33.203686 [APP/PROC/WEB/1] at com.mendix.external.connector.MxRuntimeConnector.$anonfun$processRequest$1$adapted(MxRuntimeConnector.scala:54)
2023-05-11T15:57:33.203761 [APP/PROC/WEB/1] at com.mendix.util.classloading.Runner$.withContextClassLoader(Runner.scala:20)
2023-05-11T15:57:33.203774 [APP/PROC/WEB/1] at com.mendix.external.connector.MxRuntimeConnector.processRequest(MxRuntimeConnector.scala:54)
2023-05-11T15:57:33.203787 [APP/PROC/WEB/1] at com.mendix.basis.impl.MxRuntimeImpl.processRequest(MxRuntimeImpl.scala:221)
2023-05-11T15:57:33.203821 [APP/PROC/WEB/1] at com.mendix.m2ee.appcontainer.server.handler.RuntimeServlet.service(RuntimeServlet.scala:40)
2023-05-11T15:57:33.203863 [APP/PROC/WEB/1] at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
2023-05-11T15:57:33.203931 [APP/PROC/WEB/1] at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:764)
2023-05-11T15:57:33.203969 [APP/PROC/WEB/1] at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1665)
2023-05-11T15:57:33.204029 [APP/PROC/WEB/1] at org.eclipse.jetty.websocket.servlet.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:170)
2023-05-11T15:57:33.204040 [APP/PROC/WEB/1] at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
2023-05-11T15:57:33.204092 [APP/PROC/WEB/1] at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
2023-05-11T15:57:33.204108 [APP/PROC/WEB/1] at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527)
2023-05-11T15:57:33.204162 [APP/PROC/WEB/1] at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)
2023-05-11T15:57:33.204166 [APP/PROC/WEB/1] at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1571)
2023-05-11T15:57:33.204254 [APP/PROC/WEB/1] at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)
2023-05-11T15:57:33.204304 [APP/PROC/WEB/1] at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1378)
2023-05-11T15:57:33.204365 [APP/PROC/WEB/1] at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176)
2023-05-11T15:57:33.204417 [APP/PROC/WEB/1] at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484)
2023-05-11T15:57:33.204486 [APP/PROC/WEB/1] at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1544)
2023-05-11T15:57:33.204551 [APP/PROC/WEB/1] at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174)
2023-05-11T15:57:33.204622 [APP/PROC/WEB/1] at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1300)
2023-05-11T15:57:33.204645 [APP/PROC/WEB/1] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129)
2023-05-11T15:57:33.204692 [APP/PROC/WEB/1] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
2023-05-11T15:57:33.204704 [APP/PROC/WEB/1] at org.eclipse.jetty.server.Server.handle(Server.java:562)
2023-05-11T15:57:33.204776 [APP/PROC/WEB/1] at org.eclipse.jetty.server.HttpChannel.lambda$handle$0(HttpChannel.java:505)
2023-05-11T15:57:33.204780 [APP/PROC/WEB/1] at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:762)
2023-05-11T15:57:33.204782 [APP/PROC/WEB/1] at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:497)
2023-05-11T15:57:33.204805 [APP/PROC/WEB/1] at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:282)
2023-05-11T15:57:33.204864 [APP/PROC/WEB/1] at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:319)
2023-05-11T15:57:33.204890 [APP/PROC/WEB/1] at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
2023-05-11T15:57:33.204970 [APP/PROC/WEB/1] at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
2023-05-11T15:57:33.205039 [APP/PROC/WEB/1] at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:412)
2023-05-11T15:57:33.205078 [APP/PROC/WEB/1] at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:381)
2023-05-11T15:57:33.205099 [APP/PROC/WEB/1] at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:268)
2023-05-11T15:57:33.205126 [APP/PROC/WEB/1] at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.lambda$new$0(AdaptiveExecutionStrategy.java:138)
2023-05-11T15:57:33.205166 [APP/PROC/WEB/1] at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:407)
2023-05-11T15:57:33.205169 [APP/PROC/WEB/1] at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:894)
2023-05-11T15:57:33.205205 [APP/PROC/WEB/1] at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1038)
2023-05-11T15:57:33.205234 [APP/PROC/WEB/1] at java.base/java.lang.Thread.run(Unknown Source)
2023-05-11T15:57:33.205334 [APP/PROC/WEB/1] Caused by: java.util.concurrent.TimeoutException: Idle timeout expired: 30000/30000 ms
2023-05-11T15:57:33.205345 [APP/PROC/WEB/1] at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:166)
2023-05-11T15:57:33.205389 [APP/PROC/WEB/1] at org.eclipse.jetty.io.IdleTimeout.idleCheck(IdleTimeout.java:108)
2023-05-11T15:57:33.205392 [APP/PROC/WEB/1] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
2023-05-11T15:57:33.205424 [APP/PROC/WEB/1] at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
2023-05-11T15:57:33.205451 [APP/PROC/WEB/1] at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
2023-05-11T15:57:33.205477 [APP/PROC/WEB/1] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
2023-05-11T15:57:33.205568 [APP/PROC/WEB/1] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
2023-05-11T15:57:33.205571 [APP/PROC/WEB/1] at java.base/java.lang.Thread.run(Unknown Source)
No solution had been found yet. We're muddling through with instructions to try, which work much of the time. Eventually, the documents are being uploaded, but the end user experience is not what we want.
Hi Ned,
We experienced the same error on our Production environment. And we were able to reproduce it on our ACC environment by uploading 10 PDF files of each 8,9MB with the filedropper, and afterwards quickly closing the tab in your browser.
Besides, we also see a major spike in our application node CPU usage at the same time this error occurs (on PRD as well as when we reproduced this error on ACC). Do you also see this CPU spike?