Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API CHANGE: add to_dataframe to each table #126

Open
danield137 opened this issue Apr 2, 2019 · 2 comments
Open

API CHANGE: add to_dataframe to each table #126

danield137 opened this issue Apr 2, 2019 · 2 comments

Comments

@danield137
Copy link
Contributor

danield137 commented Apr 2, 2019

After investing work in #124,
and some internal discussions, we agreed to wait with this PR and reconsider changing the API to give better performance for both vanilla python and pandas use cases, and save some difficult trickery to allow parsing kusto type to dataframe:

Final api would look like

# result is of type KustoResultDataSet
result = client.execute(db, query)
# raw json 
result.tables[0].json()
# iterator with lazy parsing of json
result.tables[0].rows()
# dataframe parsing from raw json
result.tables[0].to_dataframe()

This will cause some memory pressure, so a best practice would probably be:

# either explicitly access a specific table and drop the reference after conversion
df = client.execute(db, query).primary_results[0].to_dataframe()
# or, parse it all
dfs = client.execute(db, query).to_dataframes()

Feel free to add your thoughts, code will be implemented in next couple of weeks.

@danield137
Copy link
Contributor Author

#127

@vladikbr
Copy link

vladikbr commented Aug 5, 2020

Postponed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants