Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RSQLite (JSON functions) vs jqr/jq #28

Closed
mmuurr opened this issue Sep 19, 2020 · 2 comments
Closed

RSQLite (JSON functions) vs jqr/jq #28

mmuurr opened this issue Sep 19, 2020 · 2 comments

Comments

@mmuurr
Copy link

mmuurr commented Sep 19, 2020

Hi there, I stumbled upon your package repo while going through some issues I've found with jqr and I noticed you recently made a large commit (8f6fe42) that suggests you've decided to use RSQLite's JSON utilities instead of jqr. I'm curious about how you got to that decision (since I'm probably in a similar situation myself) and if you've done some performance evaluations of that architecture change. My impression is that jqr is quite fast, and I'm wondering if the overhead of SQLite communication adds much of a performance penalty. (I also don't know if perhaps SQLite is just using jqr -- or something similar -- under their hood, too :-).

@mgirlich
Copy link
Owner

I agree that jq is very fast and extremely powerful. But I had some issues with jq itself and the R package jqr:

I haven't done any performance tests but SQLite seemed pretty fast to me.

So, I think both options are good to work with. If you are familiar with SQL and you don't need to do overly complex operations or you rather need to work with tabular data in the end I would suggest you try using SQLite.

@mgirlich
Copy link
Owner

Btw if speed is your concern you might also have a look at the json parser rcppsimdjson.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants