Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build a WASM-compatible object_store alternative #41

Open
johnbchron opened this issue Oct 21, 2024 · 0 comments
Open

Build a WASM-compatible object_store alternative #41

johnbchron opened this issue Oct 21, 2024 · 0 comments
Assignees

Comments

@johnbchron
Copy link
Contributor

johnbchron commented Oct 21, 2024

We need to be able to access arbitrary object storage from a Cloudflare worker.

Our access pattern for the crate object_store can be found in the first-party storage crate.

Basically, we use a models::StorageCredentials to create some struct that can access the storage that our StorageCredentials are semantically pointing to. Sometimes that's local storage, in which object_store is not involved, but in the case of R2, we do use object_store. Right now, we use get() with streaming, multipart methods, and head() for testing.

The problem is that object_store use reqwest, which uses sockets for networking, i.e. the normal approach. In WASM, we can't really do that. Usually you use browser APIs, but we don't have those either. We have the Fetch API in the worker crate. This is non-standard partially because they couldn't provide browser APIs, and partially because they have to translate it back to JS and send it back through their proxy network as an outbound request.

I think it would be best if you could fork object_store to allow plugging in a different request backend. If reqwest has native support for this, that's the optimal situation. If not, it may work to make the interface sub out reqwest, though that might breach the complexity barrier. If not that, you could just recreate a minified version of object_store in the Nika repo.

@johnbchron johnbchron changed the title Build a WASM-compatible object_store alternative` Build a WASM-compatible object_store alternative Oct 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants