Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streaming API for RPC #20

Open
tejorupan opened this issue Oct 13, 2013 · 0 comments
Open

Streaming API for RPC #20

tejorupan opened this issue Oct 13, 2013 · 0 comments

Comments

@tejorupan
Copy link

Gorilla RPC does not have any provision to support streaming.

Use-case (1) is retrieving 1000s of records of data from a database using SQL, in a loop. We may not know how many rows are going to be retrieved nor we would like to store all the records in memory and do the marshalling of the whole data at once, which may not be memory efficient.

Use case (2) is to transmit bulk data, for example the contents of a file, whose size is several mega/giga bytes.

For the above mentioned cases, we avoid the RPC layer and directly work on the underlying network connection.

One interesting solution for streaming rpc is https://github.com/youtube/vitess/tree/master/go/rpcplus‎

We need a provision to process data in chunks (streaming mode) instead of waiting for the whole datastructure to be populated first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

2 participants