Optimized way to retrieve

0
Hello Mendix Experts, I want to retrieve more than 30000 records through microflow and displaying it through datagrid2 (10 records at one time), with virtual scroll and without any addition load more button. what is the optimize way to reduce loading time?   Note: I am using batch process in microflow while retrieve.
asked
1 answers
0

Hi Ankit, Good Question you want to retrieve and display 30,000+ records in DataGrid2, using virtual scroll (10 at a time), without a "Load More" button, and you’re using a microflow with batching.

 

Best Practice (If Indexing is Possible)

  1. Use Cursor-Based Batching

    • Retrieve records in small batches (e.g., 1000 at a time) using a reference field like CreatedDate or ID.

    • Example: Retrieve where ID > lastID, limit 1000

  2. Add Indexes to Improve Speed

    • Add indexes to fields used in filters or sorting (e.g., CreatedDate).

    • This makes database queries much faster.

  3. Avoid Calculated Attributes

    • Replace calculated fields with stored ones.

    • Calculated fields slow down large queries.

  4. Optimize Microflows

    • Retrieve outside loops.

    • Commit in bulk, not inside loops.

  5. Use Virtual Scroll Efficiently

    • Keep page size small (10 is good).

    • Keep columns simple, no heavy logic.

Alternatives (If Indexing is NOT Possible)

  1. Use Boolean Flags or Categories

    • Mark records you need (e.g.,IsVisible = true) and only retrieve those.

  2. Use a Separate Display Entity

    • Move the needed data into a small, simple entity just for display.

    • Speeds up grid performance.

  3. Microflow Paging with Offset

    • Retrieve data in pages manually (e.g., 0 –1000, 1001–2000, etc.).

    • Less efficient than cursor-based but works if data is stable.

  4. Client-Side Paging (For Static Data)

    • Load once into memory (non-persistent entity), then paginate on the client.

    • Only use this if the data doesn't change often.

Recommendation

If you can use indexing, do it. It gives the best performance. If you can’t use indexing, then try these:

  • Filter data with Boolean flags

  • Use simplified display entities

  • Do manual paging in microflows

  • Avoid calculated fields and complex filters

I hope this one helps you! :)

answered