From 6ef84921b08662edc13323363a9562ddfe9880d7 Mon Sep 17 00:00:00 2001 From: Gustavo Grieco <31542053+ggrieco-tob@users.noreply.github.com> Date: Wed, 22 Mar 2023 11:25:41 +0100 Subject: [PATCH 1/8] Create state-network-forking.md --- .../echidna/advanced/state-network-forking.md | 96 +++++++++++++++++++ 1 file changed, 96 insertions(+) create mode 100644 program-analysis/echidna/advanced/state-network-forking.md diff --git a/program-analysis/echidna/advanced/state-network-forking.md b/program-analysis/echidna/advanced/state-network-forking.md new file mode 100644 index 00000000..076eb8df --- /dev/null +++ b/program-analysis/echidna/advanced/state-network-forking.md @@ -0,0 +1,96 @@ +# On-chain fuzzing with state forking + +**Table of contents:** + +- [On-chain fuzzing with state forking](#on-chain-fuzzing-with-state-forking) + - [Example](#example) + - [Corpus and RPC cache](#corpus-and-rpc-cache) + - [Coverage and Etherscan integration](#coverage-and-etherscan-integration) + +## Example + +One of the most anticipated features of Echidna 2.1.0 is the state network forking. This means that Echidna can run starting with an existing blockchain state provided by an external RPC service (Infura, Alchemy, local node, etc). +This enables users to speed up the fuzzing setup when using already deployed contracts. For instance: + +```solidity +interface IHevm { + function warp(uint256 newTimestamp) external; + function roll(uint256 newNumber) external; +} + +interface Compound { + function mint() external payable; + function balanceOf(address) external view returns (uint256); +} + +contract TestCompoundEthMint { + address constant HEVM_ADDRESS = 0x7109709ECfa91a80626fF3989D68f67F5b1DD12D; + IHevm hevm = IHevm(HEVM_ADDRESS); + Compound comp = Compound(0x4Ddc2D193948926D02f9B1fE9e1daa0718270ED5); + + constructor() { + hevm.roll(16771449); // sets the correct block number + hevm.warp(1678131671); // sets the expected timestamp for the block number + } + + function assertNoBalance() public payable { + require(comp.balanceOf(address(this)) == 0); + comp.mint{value: msg.value}(); + assert(comp.balanceOf(address(this)) == 0); + } +} +``` + +This test will fail if the minting of cETH success and the balance of the contract increases. In order to use this feature, the user needs to specify the RPC endpoint for Echidna to use before running the fuzzing campaign. This requires using the following environment variables: + +``` +export ECHIDNA_RPC_URL=http://.. ECHIDNA_RPC_BLOCK=16771449 +``` + +And then Echidna can be executed as usual. When tool starts, it will start fetching bytecodes and slots as needed. +You can press the key `f` key to see which contracts/slots are fetched. + +``` +$ echidna compound.sol --test-mode assertion --contract TestCompoundEthMint +... +assertNoBalance(): failed!💥 + Call sequence, shrinking (885/5000): + assertNoBalance() Value: 0xd0411a5 +``` + +## Corpus and RPC cache + +If a corpus directory is used (e.g. `--corpus-dir corpus`), Echidna will save the fetched information inside the `cache` directory. +This will speed-up the retest of the corpus since the are no data to fetch from the RPC. It is very recommended to use this feature, in particular +if the testing is performed as part of the CI tests. + +``` +$ ls corpus/cache/ +block_16771449_fetch_cache_contracts.json block_16771449_fetch_cache_slots.json +``` + +## Coverage and Etherscan integration + +At the end of the execution, if the source code mapping of any executed on-chain contract is available on Etherscan, it will be automatically fetched for the coverage report. Optionally, an Etherscan key can be provided using the `ETHERSCAN_API_KEY` environment variable. + +``` +Fetching Solidity source for contract at address 0x4Ddc2D193948926D02f9B1fE9e1daa0718270ED5... Retrying (5 left). Error: Max rate limit reached, please use API Key for higher rate limit +Retrying (4 left). Error: Max rate limit reached, please use API Key for higher rate limit +Retrying (3 left). Error: Max rate limit reached, please use API Key for higher rate limit +Success! +Fetching Solidity source map for contract at address 0x4Ddc2D193948926D02f9B1fE9e1daa0718270ED5... Error! +``` + +While the source code for the [cETH contract is available](https://etherscan.io/address/0x4ddc2d193948926d02f9b1fe9e1daa0718270ed5#code), their source maps are NOT. +In order to generate the coverage report for fetched contract, **both** source code and source maping should be available. In such case, there will be a new +directory inside the corpus to show coverage for each contract that was fetched. All the cases, coverage is available for the user-provided contracts: + +``` +20 | | +21 | *r | function assertNoBalance() public payable { +22 | *r | require(comp.balanceOf(address(this)) == 0); +23 | *r | comp.mint{value: msg.value}(); +24 | *r | assert(comp.balanceOf(address(this)) == 0); +25 | | } +``` + From d1abd673f125d5c1cb5e7757900a6b70961a4612 Mon Sep 17 00:00:00 2001 From: Gustavo Grieco <31542053+ggrieco-tob@users.noreply.github.com> Date: Wed, 22 Mar 2023 11:27:07 +0100 Subject: [PATCH 2/8] Update README.md --- program-analysis/echidna/advanced/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/program-analysis/echidna/advanced/README.md b/program-analysis/echidna/advanced/README.md index 4642f151..c5ca86b0 100644 --- a/program-analysis/echidna/advanced/README.md +++ b/program-analysis/echidna/advanced/README.md @@ -9,3 +9,4 @@ - [How to use hevm cheats to test permit](./hevm-cheats-to-test-permit.md): How to test code that depends on ecrecover signatures using hevm cheat codes - [How to seed Echidna with unit tests](./end-to-end-testing.md): How to use existing unit tests to seed Echidna - [Understanding and using `multi-abi`](./using-multi-abi.md): What is `multi-abi` testing, and how can it be used +- [How to do state network fuzzing](./state-network-forking.md): How Echidna can use the state of blockchain during a fuzzing campaign From ea2b242c7813e66401e52250b70663f869f00b41 Mon Sep 17 00:00:00 2001 From: Gustavo Grieco <31542053+ggrieco-tob@users.noreply.github.com> Date: Tue, 28 Mar 2023 16:58:37 +0200 Subject: [PATCH 3/8] Update state-network-forking.md --- .../echidna/advanced/state-network-forking.md | 21 +++++++------------ 1 file changed, 7 insertions(+), 14 deletions(-) diff --git a/program-analysis/echidna/advanced/state-network-forking.md b/program-analysis/echidna/advanced/state-network-forking.md index 076eb8df..622e0398 100644 --- a/program-analysis/echidna/advanced/state-network-forking.md +++ b/program-analysis/echidna/advanced/state-network-forking.md @@ -41,28 +41,22 @@ contract TestCompoundEthMint { } ``` -This test will fail if the minting of cETH success and the balance of the contract increases. In order to use this feature, the user needs to specify the RPC endpoint for Echidna to use before running the fuzzing campaign. This requires using the following environment variables: +This test will fail if the call to [Compound ETH](https://etherscan.io/token/0x4Ddc2D193948926D02f9B1fE9e1daa0718270ED5) `mint` function succeeds and the balance of the contract increases. In order to use this feature, the user needs to specify the RPC endpoint for Echidna to use before running the fuzzing campaign. This requires using the `ECHIDNA_RPC_URL` and `ECHIDNA_RPC_BLOCK` environment variables: ``` -export ECHIDNA_RPC_URL=http://.. ECHIDNA_RPC_BLOCK=16771449 -``` - -And then Echidna can be executed as usual. When tool starts, it will start fetching bytecodes and slots as needed. -You can press the key `f` key to see which contracts/slots are fetched. - -``` -$ echidna compound.sol --test-mode assertion --contract TestCompoundEthMint +$ ECHIDNA_RPC_URL=http://.. ECHIDNA_RPC_BLOCK=16771449 echidna compound.sol --test-mode assertion --contract TestCompoundEthMint ... assertNoBalance(): failed!💥 Call sequence, shrinking (885/5000): assertNoBalance() Value: 0xd0411a5 ``` +Echidna will query contract code or storage slots as needed from the provided RPC node. You can press the key `f` key to see which contracts/slots are fetched. + ## Corpus and RPC cache If a corpus directory is used (e.g. `--corpus-dir corpus`), Echidna will save the fetched information inside the `cache` directory. -This will speed-up the retest of the corpus since the are no data to fetch from the RPC. It is very recommended to use this feature, in particular -if the testing is performed as part of the CI tests. +This will speed up subsequent runs, since the data does not need to be fetched from the RPC. It is recommended to use this feature, in particular if the testing is performed as part of the CI tests. ``` $ ls corpus/cache/ @@ -71,7 +65,7 @@ block_16771449_fetch_cache_contracts.json block_16771449_fetch_cache_slots.json ## Coverage and Etherscan integration -At the end of the execution, if the source code mapping of any executed on-chain contract is available on Etherscan, it will be automatically fetched for the coverage report. Optionally, an Etherscan key can be provided using the `ETHERSCAN_API_KEY` environment variable. +When the fuzzing campaign is over, if the source code mapping of any executed on-chain contract is available on Etherscan, it will be fetched automatically for the coverage report. Optionally, an Etherscan key can be provided using the `ETHERSCAN_API_KEY` environment variable. ``` Fetching Solidity source for contract at address 0x4Ddc2D193948926D02f9B1fE9e1daa0718270ED5... Retrying (5 left). Error: Max rate limit reached, please use API Key for higher rate limit @@ -82,8 +76,7 @@ Fetching Solidity source map for contract at address 0x4Ddc2D193948926D02f9B1fE9 ``` While the source code for the [cETH contract is available](https://etherscan.io/address/0x4ddc2d193948926d02f9b1fe9e1daa0718270ed5#code), their source maps are NOT. -In order to generate the coverage report for fetched contract, **both** source code and source maping should be available. In such case, there will be a new -directory inside the corpus to show coverage for each contract that was fetched. All the cases, coverage is available for the user-provided contracts: +In order to generate the coverage report for fetched contract, **both** source code and source maping should be available. In such case, there will be a new directory inside the corpus to show coverage for each contract that was fetched. All the cases, coverage is available for the user-provided contracts: ``` 20 | | From a9d0e48b0bbaf901b269aa2cfe48f7cd938f7c98 Mon Sep 17 00:00:00 2001 From: Gustavo Grieco <31542053+ggrieco-tob@users.noreply.github.com> Date: Tue, 28 Mar 2023 17:10:17 +0200 Subject: [PATCH 4/8] Update README.md --- program-analysis/echidna/advanced/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/program-analysis/echidna/advanced/README.md b/program-analysis/echidna/advanced/README.md index c5ca86b0..1f7c689f 100644 --- a/program-analysis/echidna/advanced/README.md +++ b/program-analysis/echidna/advanced/README.md @@ -9,4 +9,4 @@ - [How to use hevm cheats to test permit](./hevm-cheats-to-test-permit.md): How to test code that depends on ecrecover signatures using hevm cheat codes - [How to seed Echidna with unit tests](./end-to-end-testing.md): How to use existing unit tests to seed Echidna - [Understanding and using `multi-abi`](./using-multi-abi.md): What is `multi-abi` testing, and how can it be used -- [How to do state network fuzzing](./state-network-forking.md): How Echidna can use the state of blockchain during a fuzzing campaign +- [How to do on-chain fuzzing with state forking](./state-network-forking.md): How Echidna can use the state of blockchain during a fuzzing campaign From 2886f3916d7b8a2e68c018ab984cbbf3241c0c36 Mon Sep 17 00:00:00 2001 From: Gustavo Grieco <31542053+ggrieco-tob@users.noreply.github.com> Date: Tue, 28 Mar 2023 17:15:37 +0200 Subject: [PATCH 5/8] Update state-network-forking.md --- program-analysis/echidna/advanced/state-network-forking.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/program-analysis/echidna/advanced/state-network-forking.md b/program-analysis/echidna/advanced/state-network-forking.md index 622e0398..b8d61a1c 100644 --- a/program-analysis/echidna/advanced/state-network-forking.md +++ b/program-analysis/echidna/advanced/state-network-forking.md @@ -76,7 +76,7 @@ Fetching Solidity source map for contract at address 0x4Ddc2D193948926D02f9B1fE9 ``` While the source code for the [cETH contract is available](https://etherscan.io/address/0x4ddc2d193948926d02f9b1fe9e1daa0718270ed5#code), their source maps are NOT. -In order to generate the coverage report for fetched contract, **both** source code and source maping should be available. In such case, there will be a new directory inside the corpus to show coverage for each contract that was fetched. All the cases, coverage is available for the user-provided contracts: +In order to generate the coverage report for a fetched contract, **both** source code and source mapping should be available. In that case, there will be a new directory inside the corpus directory to show coverage for each contract that was fetched. In any case, the coverage report will be always available for the user-provided contracts, such as this one: ``` 20 | | From 336db3611332c1c5b8b171a363da313c7d9d7526 Mon Sep 17 00:00:00 2001 From: ggrieco-tob Date: Fri, 7 Apr 2023 14:25:06 +0200 Subject: [PATCH 6/8] run format --- .../echidna/advanced/state-network-forking.md | 45 ++++++++++--------- 1 file changed, 23 insertions(+), 22 deletions(-) diff --git a/program-analysis/echidna/advanced/state-network-forking.md b/program-analysis/echidna/advanced/state-network-forking.md index b8d61a1c..ae333674 100644 --- a/program-analysis/echidna/advanced/state-network-forking.md +++ b/program-analysis/echidna/advanced/state-network-forking.md @@ -9,44 +9,46 @@ ## Example -One of the most anticipated features of Echidna 2.1.0 is the state network forking. This means that Echidna can run starting with an existing blockchain state provided by an external RPC service (Infura, Alchemy, local node, etc). +One of the most anticipated features of Echidna 2.1.0 is the state network forking. This means that Echidna can run starting with an existing blockchain state provided by an external RPC service (Infura, Alchemy, local node, etc). This enables users to speed up the fuzzing setup when using already deployed contracts. For instance: ```solidity interface IHevm { function warp(uint256 newTimestamp) external; + function roll(uint256 newNumber) external; } interface Compound { - function mint() external payable; - function balanceOf(address) external view returns (uint256); + function mint() external payable; + + function balanceOf(address) external view returns (uint256); } contract TestCompoundEthMint { - address constant HEVM_ADDRESS = 0x7109709ECfa91a80626fF3989D68f67F5b1DD12D; - IHevm hevm = IHevm(HEVM_ADDRESS); - Compound comp = Compound(0x4Ddc2D193948926D02f9B1fE9e1daa0718270ED5); - - constructor() { - hevm.roll(16771449); // sets the correct block number - hevm.warp(1678131671); // sets the expected timestamp for the block number - } - - function assertNoBalance() public payable { - require(comp.balanceOf(address(this)) == 0); - comp.mint{value: msg.value}(); - assert(comp.balanceOf(address(this)) == 0); - } + address constant HEVM_ADDRESS = 0x7109709ECfa91a80626fF3989D68f67F5b1DD12D; + IHevm hevm = IHevm(HEVM_ADDRESS); + Compound comp = Compound(0x4Ddc2D193948926D02f9B1fE9e1daa0718270ED5); + + constructor() { + hevm.roll(16771449); // sets the correct block number + hevm.warp(1678131671); // sets the expected timestamp for the block number + } + + function assertNoBalance() public payable { + require(comp.balanceOf(address(this)) == 0); + comp.mint{ value: msg.value }(); + assert(comp.balanceOf(address(this)) == 0); + } } ``` -This test will fail if the call to [Compound ETH](https://etherscan.io/token/0x4Ddc2D193948926D02f9B1fE9e1daa0718270ED5) `mint` function succeeds and the balance of the contract increases. In order to use this feature, the user needs to specify the RPC endpoint for Echidna to use before running the fuzzing campaign. This requires using the `ECHIDNA_RPC_URL` and `ECHIDNA_RPC_BLOCK` environment variables: +This test will fail if the call to [Compound ETH](https://etherscan.io/token/0x4Ddc2D193948926D02f9B1fE9e1daa0718270ED5) `mint` function succeeds and the balance of the contract increases. In order to use this feature, the user needs to specify the RPC endpoint for Echidna to use before running the fuzzing campaign. This requires using the `ECHIDNA_RPC_URL` and `ECHIDNA_RPC_BLOCK` environment variables: ``` $ ECHIDNA_RPC_URL=http://.. ECHIDNA_RPC_BLOCK=16771449 echidna compound.sol --test-mode assertion --contract TestCompoundEthMint ... -assertNoBalance(): failed!💥 +assertNoBalance(): failed!💥 Call sequence, shrinking (885/5000): assertNoBalance() Value: 0xd0411a5 ``` @@ -55,7 +57,7 @@ Echidna will query contract code or storage slots as needed from the provided RP ## Corpus and RPC cache -If a corpus directory is used (e.g. `--corpus-dir corpus`), Echidna will save the fetched information inside the `cache` directory. +If a corpus directory is used (e.g. `--corpus-dir corpus`), Echidna will save the fetched information inside the `cache` directory. This will speed up subsequent runs, since the data does not need to be fetched from the RPC. It is recommended to use this feature, in particular if the testing is performed as part of the CI tests. ``` @@ -75,7 +77,7 @@ Success! Fetching Solidity source map for contract at address 0x4Ddc2D193948926D02f9B1fE9e1daa0718270ED5... Error! ``` -While the source code for the [cETH contract is available](https://etherscan.io/address/0x4ddc2d193948926d02f9b1fe9e1daa0718270ed5#code), their source maps are NOT. +While the source code for the [cETH contract is available](https://etherscan.io/address/0x4ddc2d193948926d02f9b1fe9e1daa0718270ed5#code), their source maps are NOT. In order to generate the coverage report for a fetched contract, **both** source code and source mapping should be available. In that case, there will be a new directory inside the corpus directory to show coverage for each contract that was fetched. In any case, the coverage report will be always available for the user-provided contracts, such as this one: ``` @@ -86,4 +88,3 @@ In order to generate the coverage report for a fetched contract, **both** source 24 | *r | assert(comp.balanceOf(address(this)) == 0); 25 | | } ``` - From 5cfa1bada834e8681ee9b848530936adf778c620 Mon Sep 17 00:00:00 2001 From: Gustavo Grieco <31542053+ggrieco-tob@users.noreply.github.com> Date: Tue, 2 May 2023 13:58:27 +0200 Subject: [PATCH 7/8] Update state-network-forking.md --- .../echidna/advanced/state-network-forking.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/program-analysis/echidna/advanced/state-network-forking.md b/program-analysis/echidna/advanced/state-network-forking.md index ae333674..f8e093cb 100644 --- a/program-analysis/echidna/advanced/state-network-forking.md +++ b/program-analysis/echidna/advanced/state-network-forking.md @@ -3,14 +3,18 @@ **Table of contents:** - [On-chain fuzzing with state forking](#on-chain-fuzzing-with-state-forking) + - [Introduction](#introduction) - [Example](#example) - [Corpus and RPC cache](#corpus-and-rpc-cache) - [Coverage and Etherscan integration](#coverage-and-etherscan-integration) +## Introduction + +Echidna recently added support for state network forking, starting from the 2.1.0 release. In a few words, our fuzzer can run a campaign starting with an existing blockchain state provided by an external RPC service (Infura, Alchemy, local node, etc). This enables users to speed up the fuzzing setup when using already deployed contracts. + ## Example -One of the most anticipated features of Echidna 2.1.0 is the state network forking. This means that Echidna can run starting with an existing blockchain state provided by an external RPC service (Infura, Alchemy, local node, etc). -This enables users to speed up the fuzzing setup when using already deployed contracts. For instance: +In the following contract, an assertion will fail if the call to [Compound ETH](https://etherscan.io/token/0x4Ddc2D193948926D02f9B1fE9e1daa0718270ED5) `mint` function succeeds and the balance of the contract increases. ```solidity interface IHevm { @@ -43,7 +47,7 @@ contract TestCompoundEthMint { } ``` -This test will fail if the call to [Compound ETH](https://etherscan.io/token/0x4Ddc2D193948926D02f9B1fE9e1daa0718270ED5) `mint` function succeeds and the balance of the contract increases. In order to use this feature, the user needs to specify the RPC endpoint for Echidna to use before running the fuzzing campaign. This requires using the `ECHIDNA_RPC_URL` and `ECHIDNA_RPC_BLOCK` environment variables: +In order to use this feature, the user needs to specify the RPC endpoint for Echidna to use before running the fuzzing campaign. This requires using the `ECHIDNA_RPC_URL` and `ECHIDNA_RPC_BLOCK` environment variables: ``` $ ECHIDNA_RPC_URL=http://.. ECHIDNA_RPC_BLOCK=16771449 echidna compound.sol --test-mode assertion --contract TestCompoundEthMint @@ -55,6 +59,8 @@ assertNoBalance(): failed!💥 Echidna will query contract code or storage slots as needed from the provided RPC node. You can press the key `f` key to see which contracts/slots are fetched. +Please note that only the state specified in the `ECHIDNA_RPC_BLOCK` will be fetched. If Echidna increases the block number, it is all just simulated locally but its state is still loaded from the initially set RPC block. + ## Corpus and RPC cache If a corpus directory is used (e.g. `--corpus-dir corpus`), Echidna will save the fetched information inside the `cache` directory. From 3b47e6809f0827dc54fac446ced068552bca35ca Mon Sep 17 00:00:00 2001 From: ggrieco-tob Date: Tue, 2 May 2023 14:16:01 +0200 Subject: [PATCH 8/8] npm run format --- program-analysis/echidna/advanced/state-network-forking.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/program-analysis/echidna/advanced/state-network-forking.md b/program-analysis/echidna/advanced/state-network-forking.md index f8e093cb..5e8e7dca 100644 --- a/program-analysis/echidna/advanced/state-network-forking.md +++ b/program-analysis/echidna/advanced/state-network-forking.md @@ -10,7 +10,7 @@ ## Introduction -Echidna recently added support for state network forking, starting from the 2.1.0 release. In a few words, our fuzzer can run a campaign starting with an existing blockchain state provided by an external RPC service (Infura, Alchemy, local node, etc). This enables users to speed up the fuzzing setup when using already deployed contracts. +Echidna recently added support for state network forking, starting from the 2.1.0 release. In a few words, our fuzzer can run a campaign starting with an existing blockchain state provided by an external RPC service (Infura, Alchemy, local node, etc). This enables users to speed up the fuzzing setup when using already deployed contracts. ## Example