-
Notifications
You must be signed in to change notification settings - Fork 89
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add cache_index_and_filter_blocks #7
base: master
Are you sure you want to change the base?
Conversation
thanks for the PR. it's great. Can you add some test cases as well? |
Ok. now it's in test_options.py |
Any reason this hasn't been pulled in? Looks like it also makes it possible to expose bottommost_compression which would be nice to be able to configure (FB recommends LZ4 top, ZSTD bottom). |
yup. some test cases failed, I am going to verify. |
Any news on this? According to facebook/rocksdb#3216 this might be a solution to a memory leak issue I have. |
yup. I am currently migrating all of the existing features to pybind11 branch. |
so if we set cache_index_and_filter_blocks to true, then it can solve the memory leak problem? |
@twmht Actually it looks like it was an unrelated garbage collection issue, I was creating too many new small objects in my merge operation that weren't getting collected in time. Thanks anyways. |
Take some attention to https://github.com/facebook/rocksdb/blob/6.3.fb/include/rocksdb/table.h#L61-L71 I have several databases with different sizes: 30gb 250gb 400gb 1.2tb. I doesn't have any memory issue. |
I will add merge operations to pybind11 branch in a few days. after that You can try the branch to test if it has memory leak. |
Update build.yml
Options helper classes
add cache_index_and_filter_blocks options, which is import when bloom filter