All workflows occasionally get stuck in licensed test environment and recover only after environment restart

0
Hi Mendix Community,We are experiencing a rare issue with workflows in our licensed test environment.The issue has occurred approximately 4 times during the last month. When it happens, all workflows in the application are affected at the same time.The main symptom is that the workflow is successfully started and the workflow instance exists with an In Progress status, but no User Tasks are created/assigned to the users.Normally, the same workflows work correctly and the User Tasks are created and assigned as expected.When the issue occurs:The workflow instance is successfully started and visible in the Workflow Admin Center.The workflow status remains In Progress.However, no User Tasks are created/assigned to the expected users.This happens to all workflows in the application at the same time, not just one specific workflow.The issue is not specific to one user, role, or workflow definition.The application itself continues to work.We do not experience this issue locally.Restarting the environment resolves the issue immediately, and workflows/User Tasks start working normally again.No application or workflow changes are required after the restart.The environment is a licensed test environment (licensed similarly to the production environment), not a local development runtime.Since the workflow instances are successfully created but the User Tasks are not created/processed, and since all workflows are affected simultaneously, we suspect this could be related to the Workflow Engine or Task Queue processing rather than the targeting configuration of an individual User Task.We would appreciate guidance on the following:Are there any known issues in Mendix 11.12.1 where workflow instances remain In Progress but User Tasks are not created?Could the Workflow Engine/Task Queue executor stop processing workflow activities while the rest of the application continues to function?Are there specific thread dumps, runtime metrics, or Task Queue information that we should collect before restarting the environment?Is there a way to restart or recover the Workflow/Task Queue processing without restarting the complete environment?What could explain why all workflows are affected simultaneously, while restarting the environment immediately restores normal workflow processing?The issue is difficult to reproduce because it has happened only around 3 times in one month.Environment: Licensed Test Environment Mendix version: 11.12.1 Behavior: Workflow starts → instance remains In Progress → no User Tasks are created/assigned Scope: All workflows are affected simultaneously Current workaround: Restarting the environment restores normal workflow processingHas anyone experienced similar behavior or have recommendations on what we should capture the next time it occurs before restarting the environment?
asked
1 answers
0

Hi Aaya,

The fact that all workflows stop creating User Tasks at the same time, while the application itself continues to run normally, is the key clue here.

If this were a workflow design issue, user targeting issue, or workflow definition issue, I would expect only specific workflows to fail. Since every workflow is affected simultaneously and a runtime restart immediately resolves it, I would focus on the workflow runtime processing layer (task queues/background processing) rather than individual workflow definitions.

I've seen similar symptoms where:

  • Workflow instances are created successfully.
  • Status remains In Progress.
  • No User Tasks are generated.
  • Restarting the environment causes pending workflow activities to start processing again.

In those cases, the workflow engine itself was not stopped, but the background processing responsible for advancing workflow activities was no longer progressing.

The next time it happens, before restarting, I would capture:

  • WorkflowEngine logs at Debug/Trace level
  • TaskQueue logs at Debug/Trace level
  • Runtime metrics from the environment
  • A thread dump from the runtime

I would also check whether the number of queued task queue jobs continues to increase while the incident is occurring. If workflow instances are being created but the queue responsible for advancing them is no longer being processed, that would match the behavior you're describing.

Another thing that stands out is that you're on Mendix 11.12.1. Since the issue is environment-wide and only recoverable through a restart, I'd strongly consider opening a Mendix Support ticket the next time it occurs and attaching:

  • Thread dump
  • Runtime logs
  • WorkflowEngine logs
  • TaskQueue logs
  • Example workflow instance IDs that are stuck

The combination of all workflows affected, instances created successfully, and restart immediately restores processing points more toward a runtime/workflow engine processing problem than a workflow configuration problem.

If you can capture diagnostics before the restart, Mendix Support should have a much better chance of identifying whether a workflow executor thread, task queue worker, or runtime process is becoming blocked.


answered