Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Perf] Build rocksdb with jemalloc enabled #12

Draft
wants to merge 1 commit into
base: mainnet-staging
Choose a base branch
from

Conversation

ljedrz
Copy link
Collaborator

@ljedrz ljedrz commented Jun 5, 2024

Over time we have tested various performance-related RocksDB options that were recommended in the official documentation and different case studies compiled by its users, but most of the time the effect on snarkOS memory use hasn't been significant.

One of the greatest improvements wrt node RAM figures to date has been the switch from the OS allocator to jemalloc, which is much better at handling memory fragmentation and parallel workloads.

It appears, however, that in order to utilize jemalloc within RocksDB specifically, it must be built with the seemingly undocumented jemalloc feature, which this PR does.

Filing this as a draft, as there are 2 open points:

  • we are currently testing it on a client in the IsoNet, but we'll also need to test it under higher load and with a validator
  • jemalloc is currently included only for x86_86 Linux, so we may (on systems without jemalloc it might just get ignored) need to tweak some TOMLs in order for it to work on other operating systems

Signed-off-by: ljedrz <ljedrz@users.noreply.github.com>
@vicsn
Copy link
Collaborator

vicsn commented Jul 24, 2024

Consider making jemalloc conditional on the architecture, just like our global jemalloc configuration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants