I want to retrieve a large amount of data (100k+ Datapoints) from SAP HANA with the Odata connector and save it in an entity. Pulling all the data at once either takes extremly long or just chrashes. Doing batches with a batch size of 2k-3k seems to be a bit faster, but is sometimes very unreliable. Is there any better solution or is maybe even using the odata connector not the best option? Thanks in advance, Joshua
asked
Joshua Schicht
1 answers
0
Batching is definitely better. I would recommend doing the sync at daily intervals for those records that frequently change and a separate scheduler to sync all data at a weekly interval.