Elasticsearch Dask Dataframe

Hi guys,

Hope you all feel well, I’m a newbie in DASK. I have trouble In creating Dataframes with read_sql_query/read_sql_table with Elasticsearch indexes. Normally for an SQL-based table, it’s easy to fetch data as Dask Dataframe directly but in Elasticsearch it’s a little bit vague.

Thank you all.

Hi @Behzad, welcome here,

I’m not sure what you are asking here. Elasticsearch is not an SQL database, so it feels normal it won’t work out of the box with these methods. Elasticsearch APIs returns JSON documents, so I’ll guess you’ll have to build something on your own.

Doing a quick Internet search, I’ve found several resources:

Maybe one of those could help you?

Starting from scratch, you should use the Elasticsearch Python client, get a JSON, and create a Dataframe from it, either parsing it, or maybe using Dask Bags.