Skip to content

Release 2.9.0-early-adopter

Latest
Latest
Compare
Choose a tag to compare
@JamesPiechota JamesPiechota released this 13 Dec 00:38
· 0 commits to master since this release
N.2.9.0-early-adopter
0d46cc9

Arweave 2.9.0-Early-Adopter Release Notes

This Arweave node implementation proposes a hard fork that activates at height 1602350, approximately 2025-02-03 14:00 UTC. This software was prepared by the Digital History Association, in cooperation with the wider Arweave ecosystem. Additionally, this release was audited by NCC Group.

This 2.9.0 release is an early adopter release. If you do not plan to benchmark and test the new data format, you do not need to upgrade for the 2.9 hard fork yet.

Note: with 2.9.0 when enabling the randomx_large_pages option you will need to configure 5,000 HugePages rather than the 3,500 required for earlier releases.

Replica 2.9 Packing Format

The Arweave 2.9.0-early-adopter release introduces a new data preparation (‘packing’) format. Starting with this release you can begin to test out this new format. This format brings significant improvements to all of the core metrics of data preparation.

To understand the details, please read the full paper here: https://github.com/ArweaveTeam/arweave/blob/release/N.2.9.0-early-adopter/papers/Arweave2_9.pdf

Additionally, an audit of this mechanism was performed by NCC group and is available to read here (the comments highlighted in this audit have since been remediated): https://github.com/ArweaveTeam/arweave/blob/release/N.2.9.0-early-adopter/papers/NCC_Group_ForwardResearch_E020578_Report_2024-12-06_v1.0.pdf

Arweave 2.9’s format enables:

  • Allow miners to read from their drives at a rate of 5 MiB/s (the equivalent of difficulty 10 in Arweave 2.8), without adversely affecting the security of the network. This represents a decrease of 90% from Arweave 2.8, and 97.5% from Arweave 2.7.x. This will allow miners to use the most cost efficient drives in order to participate in the network, while also lowering pressure on disk I/O during mining operations.
  • A ~96.9% decrease in the compute necessary to pack Arweave data when compared to 2.8 composite.1, and SPoRA_2.6. This decrease also scales approximately linearly for higher packing difficulties. For example, for miners that would have packed with Arweave 2.8 to the difficulty necessary to reach a 5 MB/s read speed (composite.10), Arweave 2.9 will require ~99.56% less energy and time. This represents an efficiency improvement of 32x against 2.7.x and 2.8 composite.1, and ~229x for composite.10.

Replica 2.9 Benchmark Tool

If you'd like to benchmark the performance of the new Replica 2.9 packing format on your own machine you can use the new ./bin/benchmark-2.9 tool. It has 2 modes:

  • Entropy generation which generates and then discards entropy. This allows you to benchmark the time it takes for your CPU to perform the work component of packing, ignoring any IO-related effects.
    • To use the entropy generation benchmark run the tool without using any dir flags.
  • Packing which generates entropy, packs some random data, and then writes it to disk. This provides a more complete benchmark of the time it might take your server to pack data. Note: This benchmark does not include unpacking or reading data (and associated disk seek times).
    • To use the packing benchmark mode specify one or more output directories using the multi-use dir flag.
Usage: benchmark-2.9 [format replica_2_9|composite|spora_2_6] [threads N] [mib N] [dir path1 dir path2 dir path3 ...]

format: format to pack. replica_2_9, composite.1, composite.10, or spora_2_6. Default: replica_2_9.
threads: number of threads to run. Default: 1.
mib: total amount of data to pack in MiB. Default: 1024.
     Will be divided evenly between threads, so the final number may be
     lower than specified to ensure balanced threads.
dir: directories to pack data to. If left off, benchmark will just simulate
     entropy generation without writing to disk.

Repacking to Replica 2.9

As well as allowing you to run benchmarks, the 2.9.0-early-adopter release also allows you to pack data for the 2.9 format. It has not, however, been fully optimized and tuned for the new entropy distribution scheme. It is included in this build for validation purposes. In our tests, we have observed consistent >=75% reductions in computation requirements (>4x faster packing speeds), but future releases will continue to improve this towards the performance of the benchmarking tool.

To test this functionality run a node with storage modules configured to use the <address>.replica.2.9 packing format. repack_in_place is not yet supported.

Composite Packing Format Deprecated

The new packing format was discovered as a result of researching an issue (not endangering data, tokens, or consensus) that affects higher difficulty packs of the 2.8 composite scheme. Given this, and the availability of the significantly improved 2.9 packing format, as of block height 1642850 (roughly 2025-04-04 14:00 UTC), data packed to any level of the composite packing format will not produce valid block solutions.

Note: This is an "Early Adopter" release. It implements significant new protocol improvements, but is still in validation. This release is intended for members of the community to try out and benchmark the new data preparation mechanism. You will not need to update your node for 2.9 unless you are interested in testing these features, until shortly before the hard fork height at 1602350 – approximately Feb 3, 2025. As this release is intended for validation purposes, please be aware that there is a possibility that data encoded using its new preparation scheme may need to be repacked before 2.9 activates. The first ‘mainline’ releases for Arweave 2.9 will follow in the coming weeks after community validation has been completed.

Full Changelog: N.2.8.3...N.2.9.0-early-adopter

Assets 8
Loading