Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prototype a RocksDB-based storage backend #5425

Closed
wants to merge 28 commits into from
Closed

Conversation

ptrus
Copy link
Member

@ptrus ptrus commented Nov 2, 2023

TODO:

  • ensure all E2E tests pass using rocks-db backend
  • prepare and run some benchmarks
  • do some long runs comparing rocksdb vs badger (performance, storage usage, ensure consistency)
  • Followup PR's: also add support for CometBFT-DB backed by rocksdb (but actually the upstream one uses it already, could just use that?)
  • Local storage rocksdb impl

@ptrus ptrus force-pushed the ptrus/feature/rocksdb branch 10 times, most recently from 492b8ef to 010fd2f Compare November 3, 2023 13:44
Copy link

codecov bot commented Nov 4, 2023

Codecov Report

Attention: 170 lines in your changes are missing coverage. Please review.

Comparison is base (3f2f349) 66.83% compared to head (3f69784) 66.65%.
Report is 20 commits behind head on master.

❗ Current head 3f69784 differs from pull request most recent head fba7342. Consider uploading reports for the commit fba7342 to get more accurate results

Files Patch % Lines
go/storage/mkvs/db/rocksdb/rocksdb.go 78.66% 65 Missing and 50 partials ⚠️
go/storage/mkvs/db/rocksdb/batch.go 82.67% 12 Missing and 10 partials ⚠️
go/storage/mkvs/db/rocksdb/iterator.go 69.23% 11 Missing and 5 partials ⚠️
go/storage/mkvs/db/rocksdb/metadata.go 85.18% 4 Missing and 4 partials ⚠️
go/storage/mkvs/node/hash.go 78.57% 3 Missing ⚠️
go/storage/mkvs/db/rocksdb/timestamp.go 95.65% 1 Missing and 1 partial ⚠️
go/consensus/cometbft/full/archive.go 0.00% 1 Missing ⚠️
go/storage/mkvs/db/api/api.go 0.00% 1 Missing ⚠️
go/storage/mkvs/db/badger/migrate.go 94.11% 1 Missing ⚠️
go/worker/storage/config/config.go 50.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5425      +/-   ##
==========================================
- Coverage   66.83%   66.65%   -0.19%     
==========================================
  Files         534      538       +4     
  Lines       56658    57172     +514     
==========================================
+ Hits        37870    38108     +238     
- Misses      14348    14601     +253     
- Partials     4440     4463      +23     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ptrus ptrus force-pushed the ptrus/feature/rocksdb branch 3 times, most recently from a4c796f to 3f69784 Compare November 10, 2023 20:59
@ptrus ptrus force-pushed the ptrus/feature/rocksdb branch from 3f69784 to 3e0d377 Compare December 5, 2023 09:40
@ptrus ptrus force-pushed the ptrus/feature/rocksdb branch 2 times, most recently from 78c8133 to c13dcb8 Compare December 15, 2023 09:01
@ptrus ptrus force-pushed the ptrus/feature/rocksdb branch from c13dcb8 to fba7342 Compare December 19, 2023 09:30
@ptrus ptrus closed this Apr 9, 2024
@ptrus ptrus deleted the ptrus/feature/rocksdb branch April 9, 2024 08:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant