Community Commons 2.2: Easy batching for large datasets
4
Hi All, Community Commons 2.2 have just been released, including several improvements and fixes filed in bug reports. But I think one function is especially interesting as might ease the creation of data migration microflows and scheduled events on large datasets, so I want to highlight it here: executeMicroflowInBatches This function takes an xpath query and a microflow, and invokes the microflow for each object that matches the query. However, this is done in small batches so the dataset never needs to be entirely in memory, and changes will be committed to the database after each batch, so that the database transactions will not be too large. The full documentation of the method: Invokes a microflow in batches. The microflow is invoked for each individual item returned by the xpath query. The objects will be processed in small batches (based on the batchsize), which makes this function very useful to process large amounts of objects without using much memory. All defaut behavior such as commit events are applied as defined in your microflow. Parameters: xpath: Fully qualified xpath query that indicates the set of objects the microflow should be invoked on. For example: '//System.User[Active = true()]' microflow: The microflow that should be invoked. Should accept one argument of the same type as the xpath. For example: 'MyFirstModule.UpdateBirthday' batchsize: The amount of objects that should be processed in a single transaction. When in doubt, 1 is fine, but larger batches (for example; 100) will be faster due to less overhead. waitUntilFinished: Whether this call should block (wait) until all objects are processed. The function returns true if the batch has successfully started, or, if waitUntilFinished is true, returns true if the batch succeeded completely. Note, if new objects are added to the dataset while the batch is still running, those objects will be processed as well. Want to contribute to the community commons project? Send your addons to, or request a project invitation at community@mendix.com