Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

more complete async support and async bug #578

Open
matthewgapp opened this issue Jul 13, 2024 · 2 comments
Open

more complete async support and async bug #578

matthewgapp opened this issue Jul 13, 2024 · 2 comments

Comments

@matthewgapp
Copy link

So I got arrow-odbc (almost) working with async-polling with these two PRs: pacman82/arrow-odbc#102 and #577

But I'm running into an issue with the snowflake ODBC driver. We're using the snowflake driver, which complains that we free the statement before we've completed executing (I've concluded that based on the error message and the MS docs here (https://learn.microsoft.com/en-us/sql/odbc/reference/syntax/sqlfreestmt-function?view=sql-server-ver16#diagnostics).

The query seems to stream fine, but the driver fails when the inner BlockCursorPolling is dropped in the AsyncBatchedOdbcReader struct, panicking in the drop impl for BlockCursorPolling with the following error:

 Unexpected error unbinding columns: Diagnostics { record: State: HY010, Native error: 0, Message: [unixODBC][Driver Manager]Function sequence error, function: "SQLFreeStmt" }

I'm unsure how this is possible since all methods that use wait_for() do not return until StillExecuting is not returned. And if a method didn't handle StillExecuting, then it would panic here https://github.com/nobie-org/odbc-api/blob/a20309759e83ebdfbeac5c115b24b97917d6b5fd/odbc-api/src/error.rs#L244

@pacman82
Copy link
Owner

Hello @matthewgapp ,

thanks for your effort. I took the liberty and already merged a fix for the missing enablement of async in execute polling you mentioned in #577. Please take a look at the tests and note that most drivers actually do not support polling.

It was mostly this poor support for polling (not even starting to think about event based async on windows) within ODBC drivers, which made me write ConcurrentBlockCursor which uses system threads and works reliably with many drivers.

For your use-case it seems to me worth verifying that the snowflake driver does indeed support polling.

Currently I am heavily occupied by my day job, so my ability to review large pull requests is inhibited. Splitting things out into smaller features, with their own test cases would help me. I have some further thoughts, but (again) I think the first thing to check if this is indeed actually helping you at all and snowflake supports this behavior.

Best, Markus

@pacman82
Copy link
Owner

FYI: I just verified this is still working: If you want to execute all the tests locally you can just open it remotely in vscode in the devcontainer. It will boot up all databases and install all drivers in a devcontainer, provided you have docker compose installed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants