Hey Angelo,

When working with large amounts of data, it is important to break down the process into smaller, more manageable tasks. This helps avoid long processing times and system overload.

You can create microflows that process data in batches, instead of trying to do it all at once. Here is an example of how this can be done.

 1. Create a temporary entity to store the initially imported data. Use Excel Importer or Advanced Excel to import the data into this temporary entity.

2. Create a main microflow that will be responsible for starting batch processing. This main microflow will call a sub-microflow responsible for processing a batch of data.

 3. Create a microflow that fetches a batch of records from the temporary entity. 

4. Pass these records to a sub-microflow that processes each record individually. Update the status of processed records to avoid reprocessing.

This microflow must perform the necessary processing on each record, such as validations, transformations and saving to the final entity.

Consider using scheduled events to spread the processing load over time. This prevents server overload and improves scalability.

Best Regards,

Ricardo Pereira

Use the Excel importer from the marketplace, which is downloaded 100k+ times.

If that fails because your excel file is too big or the import too slow, then try to also use the Mass upload from the marketplace, which is improves on Excel importer.