Release 2.6.2

The release introduces various improvements and bug fixes.

Some hightlights:

• The memory consumption issues of 2.6.1 are fixed;
• the v2_index_data_size_by_packing metric is recorded with the new "partition_size" and "partition_index" labels;
• the node may now pack and update in place the unpacked data in the 2.5 storage (activate by enable legacy_storage_unpacked_packing);
• the default packing rate is now chosen based on the CPU core count, albeit conservatively;
• the mempool processing is improved by filtering out unfitting transactions early in the process.

The release comes with the prebuilt binaries for the Linux x86_64 platforms.

If you want to run the miner from the existing Git folder, upgrade Erlang to version 24 and execute the following command:

git fetch --all --tags && git checkout -f N.2.6.2

See the mining guide for further instructions.

If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the Arweave Miners Discord (https://discord.gg/GHB4fxVv8B) or email us at team@arweave.org.

Release 2.6.1

The release introduces a convenient way to copy and repack data across the local storage modules, including the legacy 2.5 storage.

Some hightlights:

• the data is copied and repacked from the 2.5 storage to the corresponding storage modules, if any, and between the storage modules; once finished, the node attempts to sync the missing data from the network;
• additionally, the 2.5 data is repacked in place when the node is launched with enable legacy_storage_repacking;
• the broken chunk index records are repaired in the process;
• the limits for the number of chunks to keep in RAM (by the syncing, packing, and mining processes accordingly) are chosen based on the available memory and reported on startup; the limits may be overriden via data_cache_size_limit [number], packing_cache_size_limit [number], and mining_server_chunk_cache_size_limit [number];
• the estimated maximum packing rate is reported on startup (you might want to restart the node with packing_rate [number] afterwards.

The release comes with the prebuilt binaries for the Linux x86_64 platforms. arweave-2.6.1.linux-x86_64.tar.gz works on Ubuntu 21+ and CentOS 8. To run on Ubuntu 18 or 20, grab arweave-2.6.1.ubuntu18-x86_64.tar.gz or arweave-2.6.1.ubuntu20-x86_64.tar.gz . CentOS 7 is not supported.

If you want to run the miner from the existing Git folder, upgrade Erlang to version 24 and execute the following command:

git fetch --all --tags && git checkout -f N.2.6.1

See the updated mining guide for further instructions.

If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the Arweave Miners Discord (https://discord.gg/GHB4fxVv8B) or email us at team@arweave.org.

Release 2.6

This release upgrades the network inline with the Arweave 2.6 specification. While the specification details the major upgrades to the system, some of the highlights include:

• Significantly lessened energy consumption during the mining process.
• Oracle-free real-time detection of storage costs.
• Increased capabilities to detect network capacity.

Please read the spec and mining guide for full details of these features

Transition and Implementation Details

The fork activates at height 1132210, approximately 2023-03-06 14:00 UTC. You will need to make sure you have upgraded your miner before this time to connect to the network.

The release comes with the prebuilt binaries for the Linux x86_64 platforms. arweave-2.6.0.0.linux-x86_64.tar.gz works on Ubuntu 21+ and CentOS 8. To run on Ubuntu 18 or 20, grab arweave-2.6.0.0.ubuntu18-x86_64.tar.gz or arweave-2.6.0.0.ubuntu20-x86_64.tar.gz . CentOS 7 is not supported.

If you want to run the miner from the existing Git folder, upgrade Erlang to version 24 and execute the following command:

git fetch --all --tags && git checkout -f N.2.6.0

See the updated mining guide for further instructions.

The new protocol comes with the new mechanism for determining upload fees and mining rewards. The new upload fee computation mechanism kicks in in 75 days after the fork activation and begins a gradual transition to the new pricing mechanism taking 12 months. Fiat price stabilization will be disabled during these 3 months until the transition begins due to high uncertainty about the network difficulty at the time. An additional hard fork adjusting the benchmark rate during this interim period is likely.

This release also adds mechanisms for potential situations in which the endowment value reaches extreme values. In the case that the endowment consumes almost all of the tokens in the system (<1mAR left outside the endowment), all tokens will automatically re-denominate by 3 orders of magnitude. This allows the tokens to continue to be usefully subdivided, despite the deflation of its supply. In the event that the endowment runs too low (signalling that the core parameter of a Kryder+ rate expectation of 0.5% is too optimistic), the network would halve the expected Kryder+ rate. This halving would repeat until a Kryder+ value is found that leads to the anticipated long-term deflation, while storage for old data continues to be honoured.

This release also includes a delay in the release of block rewards for miners by approximately thirty days (30 * 24 * 30 blocks). This mechanism is designed to incentivize miners to be aligned with the network for atleast the medium-term. The delay will start at the moment that the hard fork activates.

Complete weave replicas are required for maximizing the mining performance. Nevertheless, miners with small datasets can pool together by using a shared mining address.

If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the Arweave Miners Discord (https://discord.gg/GHB4fxVv8B) or email us at team@arweave.org.

Release 2.5.3

This release adds support for byte-range content policies, including compatibility with the NSFW list produced by the shepherd program. You can load that list (or any others) by adding the following to your miner's start command:

transaction_blacklist_url http://shepherd-v.com/list.txt.

For more details, see the note in the mining guide.

Upgrade instructions

N.2.5.3 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms. arweave-2.5.3.linux-x86_64.tar.gz works at least on Ubuntu 20 and CentOS 8. To run on Ubuntu 18, grab arweave-2.5.3.ub18-x86_64.tar.gz. CentOS 7 is not supported.

Download the corresponding archive and unpack it:

tar -xzf arweave-2.5.3.0.linux-x86_64.tar.gz

It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:

bin/  data/  erts-10.3/  lib/  releases/

To run the miner, execute ./bin/start . See the mining guide for more details.

If your OS/platform architecture is not on the list, check README for how to build the miner from sources.

If you want to run the miner from the existing Git folder, upgrade Erlang to version 23 or 24 and execute the following command:

git fetch --all --tags && git checkout -f N.2.5.3

You can now run the miner using the arweave-server script.

Support

For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the Arweave Miners Discord (https://discord.gg/GHB4fxVv8B) or email us at team@arweave.org.

Release 2.5.2

The release introduces various improvements aiming to improve peering, block propagation, and data synchronization. Also, a bug is fixed that would reduce the node's mining performance.

• Blocks and transactions are now served, gossiped, and fetched in the binary format (between the upgraded nodes). It reduces ~30% of the communicated traffic and speeds up processing;
• chunks are also served in the new binary format;
• the node sends compact block announcements to inform upgraded peers about the new blocks. The protocol allows the recipient to notify the sender of the missing transactions early and receive them along with the block header; also, the recipient may ask the sender to not include the recall chunk and fetch it themselves;
• a block_gossip_peer configuration parameter is added to give miners the ability to specify priority block recipients manually;
• blocks and transactions are propagated to the peers from the unique subnetworks first;
• added GET /hash_list/[from]/[to]. The endpoint serves a JSON list of Base64Url encoded hashes of the blocks between the given heights (inclusive), sorted from latest to earliest;
• processing chunks is made more efficient.

API changes

• New endpoints: GET /hash_list/[from]/[to], POST /block2, POST /tx2, GET /tx2/[id], GET /block2/hash/[hash], GET /block2/height/[height], GET /chunk2/[offset], GET /block_index2, GET /recent_hash_list_diff.

Upgrade instructions

N.2.5.2 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms. arweave-2.5.2.linux-x86_64.tar.gz works at least on Ubuntu 20 and CentOS 8. To run on Ubuntu 18, grab arweave-2.5.2.ub18-x86_64.tar.gz. CentOS 7 is not supported.

Download the corresponding archive and unpack it:

tar -xzf arweave-2.5.2.0.linux-x86_64.tar.gz

It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:

bin/  data/  erts-10.3/  lib/  releases/

To run the miner, execute ./bin/start . See the mining guide for more details.

If your OS/platform architecture is not on the list, check README for how to build the miner from sources.

If you want to run the miner from the existing Git folder, upgrade Erlang to version 23 or 24 and execute the following command:

git fetch --all --tags && git checkout -f N.2.5.2

You can now run the miner using the arweave-server script.

Support

For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the Arweave Miners Discord (https://discord.gg/GHB4fxVv8B) or email us at team@arweave.org.

Release 2.5.1.0

The release introduces a rework of the mempool management and propagation, a small miner optimization, and improvements of the syncing process, especially end game syncing and syncing of a large number of disconnected intervals.

• Transactions are not rejected when the mempool limit is reached; instead, the least valuable transactions are dropped from the memory pool;
• picking transactions for a block is made faster;
• the peers for gossip are chosen from the entire network (some preference still goes to the better-rated peers) to avoid propagation issues when the rating network becomes too clustered; blocks are also sent to the trusted peers (specified on startup);
• the default number of peers transactions are gossiped to is reduced to speed up propagation;
• the node does not only search for confirmed transactions when searching for missing transactions for a block;
• freshly synced block and transaction headers are stored in RocksDB now;
• a bug is fixed where already known transactions would only be immediately ignored on repeated submission with 5 seconds since the previous submission. At the same time, the identifiers are not kept in memory indefinitely, preventing unnecessary albeit slow increase of the memory footprint;
• the headers are always stored in the disk cache (the size is controlled by the disk_cache_size_mb parameter) to make v1 transactions with data quickly available for download before their data is picked up from the disk pool and indexed for mining;
• the redundant signature validation during mempool filtering is removed;
• the miner is accelerated ~5-10% by removing one extra in-memory chunk copy per mined chunk;
• upgraded erlang-rocksdb to 1.7.0; RocksDB 6.25.3;
• made various improvements to the syncing process;
• fixed tracking of the disk_pool_chunks_count metric;
• improved the disk pool processing in the presence of many repeated data_roots throughout the weave.

API changes

• The node will reject a v2 transaction with the attached data if the node fails to reconstruct its data_root from the data. It does not pose consensus issues because 1) nodes strip data before gossip 2) the data field is ignored when a node fetches a missing transaction itself. The motivation is to reduce the likelihood of not uploading the data by mistake, e.g., when a new client library uses this way of uploading but does not split the data accordingly.

Upgrade instructions

N.2.5.1.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms. arweave-2.5.1.0.linux-x86_64.tar.gz works at least on Ubuntu 20 and CentOS 8. To run on Ubuntu 18, grab arweave-2.5.1.0.ubuntu18-x86_64.tar.gz. CentOS 7 is not supported.

Download the corresponding archive and unpack it:

tar -xzf arweave-2.5.1.0.linux-x86_64.tar.gz

It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:

bin/  data/  erts-10.3/  lib/  releases/

To run the miner, execute ./bin/start . See the mining guide for more details.

If your OS/platform architecture is not on the list, check README for how to build the miner from sources.

If you want to run the miner from the existing Git folder, upgrade Erlang to version 23 or 24 and execute the following command:

git fetch --all --tags && git checkout -f N.2.5.1.0

You can now run the miner using the arweave-server script.

Support

For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.

Release 2.5.0.0

This release introduces a hard fork that activates at height 812970, approximately 2021-11-17 14:00 UTC. You will need to make sure you have upgraded your miner before this time to connect to the network.

This protocol upgrade includes a number of improvements, laying the groundwork for the switch to HDD-dominated mining in 2.6. After the fork, SPoRA solution candidates have to be "packed" such that they can be identified as part of a unique replica. Additionally, the protocol will now create some entropy on every mining attempt. Miners have to carry it on throughout the process, further strengthening the requirement to keep data close to the computation power.

Note for the miners: the upgrade does not make hard drives the most cost-efficient medium (we plan to make this transition in the next major release).

Note for the miners: the new mining algorithm operating with the packed data and entropy does not take effect immediately after the fork but gradually replaces the existing algorithm over time.

Note for the users: starting from the fork block, the protocol would no longer recognize new data that is not split in a particular way. Specifically, all chunk sizes have to equal 256 * 1024 bytes unless they meet one of the three requirements:

• the chunk is the last or the only chunk of its transaction;
• the chunk is the second last chunk of its transaction, and together with the last chunk, they exceed 256 KiB;
• the chunk size is larger than its Merkle proof (data_path).

The change is compatible with the major ecosystem clients, like arweave-js, arweave-stream-tx, or goar.

Note that it would still be possible to submit a transaction with a data_root computed from a wrong split, but the chunks not meeting the criteria mentioned above won't be accepted by the nodes.

Furthermore, every TX requires a fee covering the smallest multiple of 256 KiB bigger than or equal to its data_size after the fork.

The difficulty-based price adjustment is re-enabled in 2.5. The price is recalculated every 50 blocks; however, we introduce a one-time change cap to eliminate the effects of short-term difficulty and token price fluctuations.

Furthermore, an accelerated difficulty adjustment is used at the fork block, similar to the mechanism used at 2.4, because a significant network difficulty drop is possible. However, the transition is gradual, so we expect a relatively small difficulty change. The node will migrate the historical data and pack the incoming chunks to have them ready for mining.

Finally, the floating-point math is no longer present in the protocol after the fork, simplifying alternative clients' development.

Upgrade instructions

N.2.5.0.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms. arweave-2.5.0.0.linux-x86_64.tar.gz works at least on Ubuntu 20 and CentOS 8. To run on Ubuntu 18, grab arweave-2.5.0.0.ubuntu18-x86_64.tar.gz. CentOS 7 is not supported.

Download the corresponding archive and unpack it:

tar -xzf arweave-2.5.0.0.linux-x86_64.tar.gz

It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:

bin/  data/  erts-10.3/  lib/  releases/

To run the miner, execute ./bin/start . See the mining guide for more details.

If your OS/platform architecture is not on the list, check README for how to build the miner from sources.

If you want to run the miner from the existing Git folder, upgrade Erlang to version 23 or 24 and execute the following command:

git fetch --all --tags && git checkout -f N.2.5.0.0

You can now run the miner using the arweave-server script.

Support

For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.

Release 2.4.4.0

The release aims to eliminate congestion in the transaction propagation queues. Nodes now monitor eachother's mempools, eliminating a significant amount of the communication during transaction distribution.

Additionally, the upgrade speeds up the downloading of missing transactions from new blocks (an event which has increased in frequency during the transaction distribution issues).

Also, fixes #304.

Upgrade instructions

N.2.4.4.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms. arweave-2.4.4.0.linux-x86_64.tar.gz works at least on Ubuntu 20 and CentOS 8. To run on Ubuntu 18 grab arweave-2.4.4.0.ubuntu18-x86_64.tar.gz. CentOS 7 is not supported.

Download the corresponding archive and unpack it:

tar -xzf arweave-2.4.4.0.linux-x86_64.tar.gz

It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:

bin/  data/  erts-10.3/  lib/  releases/

To run the miner, execute ./bin/start . See the mining guide for more details.

If your OS/platform architecture is not in the list, check README for how to build the miner from sources.

If you want to run the miner from the existing Git folder, upgrade Erlang to version 23 and execute the following command:

git fetch --all --tags && git checkout -f N.2.4.4.0

You can now run the miner using the arweave-server script.

Support

For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.

Release 2.4.3.0

The release introduces a new reliable way to maintain the storage under the limited disk space, fixes a bug where the node would accumulate redundant chunks in the disk pool, improves transaction propagation, and contains a few smaller performance improvements.

Upgrade instructions

N.2.4.3.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms. arweave-2.4.3.0.linux-x86_64.tar.gz works at least on Ubuntu 20 and CentOS 8. To run on Ubuntu 18 grab arweave-2.4.3.0.ubuntu18-x86_64.tar.gz. CentOS 7 is not supported.

Download the corresponding archive and unpack it:

tar -xzf arweave-2.4.3.0.linux-x86_64.tar.gz

It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:

bin/  data/  erts-10.3/  lib/  releases/

To run the miner, execute ./bin/start . See the mining guide for more details.

If your OS/platform architecture is not in the list, check README for how to build the miner from sources.

If you want to run the miner from the existing Git folder, upgrade Erlang to version 23 or 24 and execute the following command:

git fetch --all --tags && git checkout -f N.2.4.3.0

You can now run the miner using the arweave-server script.

Support

For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.

Release 2.4.2.0

The release features a significant syncing speedup, various other improvements, and changes made in preparation for the new iteration of the mining algorithm.

Among the prepared but not yet activated protocol upgrades is removing floating-point arithmetic from the protocol - likely the most significant obstacle on the path to alternative node implementations.

Please, note you can run the new release with existing data, but you cannot run the previous versions of the miner from the same data_dir after running the latest release.

Other changes include:

• Synchronization is made much faster by removing the bottleneck added in 2.1.
• A header_sync_jobs command line and config file parameter is introduced for speeding up chain headers download.
• The joining process is made faster.
• A new synchronization scheme is implemented improving data discovery after a significant amount of data is synced.
• The requirement for the peers specified on startup to be available is dropped.
• v1 transactions with more than 100 bytes of data are deprioritized over all other transactions. Such transactions saturate network links and fill up blocks much quicker. Users are expected to submit v2 transactions to upload data.
• Orphaned transactions are returned to the mempool now. Very roughly, about 0.5% of all submitted transactions used to be dropped from the weave after a reorg. The measure will eliminate almost all of such cases.
• A bug is fixed where the peer list update may get stuck (the ar_bridge_update_peers_process_is_stuck log).
• A bug is fixed where the node may store and report as synced but not serve some chunks after shutdown.
• A bug is fixed where a node might lose a chunk after a chain reorg.

Upgrade instructions

N.2.4.2.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.

Download the corresponding archive and unpack it:

tar -xzf arweave-2.4.2.0.linux-x86_64.tar.gz

It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:

bin/  data/  erts-10.3/  lib/  releases/

To run the miner, execute ./bin/start . See the mining guide for more details.

If your OS/platform architecture is not in the list, check README for how to build the miner from sources.

If you want to run the miner from the existing Git folder, upgrade Erlang to version 23 and execute the following command:

git fetch --all --tags && git checkout -f N.2.4.2.0.0

You can now run the miner using the arweave-server script.

Support

For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.