Hello,
I'm pulling data into Power BI using the Elastic Stack REST API, but I'm running into a problem when trying to pull more than 10k records.
In order to do so, I need to use the scroll function: https://www.elastic.co/guide/en/elasticsearch/reference/master/search-request-scroll.html
The first call:
GET /_search?scroll=1m { "query": ... }
Will return the first set of results plus a scroll id:
DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4...
Which is used for subsequent calls:
GET /_search { "scroll_id": "DXF1Z..." }
Each call returns a set of records plus a new scroll id. The scroll id is then passed onto the next record, and so on.
For performance reasons, I don't want to increase the number of records that can be returned by the REST API.
Is it possible to solve this using Power BI?
Thanks,
Ken