John Vester
If you’re working with smart contracts—or even just exploring them—you probably already know that smart contract security is important. Smart contracts are immutable once deployed, and often involve significant amounts of money. Writing safe and reliable code before deployment should be top of mind. And as the adoption of blockchain accelerates, ensuring the security of smart contracts becomes even more important.
One of the best additions to your smart contract audit is fuzzing, a dynamic testing technique that exposes vulnerabilities by generating and injecting random inputs into your smart contracts during testing.
In this article, we’ll explore how to use fuzzing to effectively audit a smart contract. Specifically, we’ll look at ConsenSys Diligence Fuzzing—a new fuzzing as a service (FaaS) offering. We’ll delve into the technical aspects and show some code examples.
What is Fuzzing?
Fuzzing is a dynamic testing technique where random (or semi-random) inputs called “fuzz” are generated and injected into code. Fuzzing can help reveal bugs and vulnerabilities that weren’t caught by traditional testing methods.
Manual (unit) testing requires you to figure out what functionality to test, what inputs to use, and what the expected output should be. It’s time-consuming, difficult, and in the end, it’s still easy to miss scenarios.
On the other hand, fuzzing (or fuzz testing) is an automated testing process that sends random data into an application to test its security. A fuzzer can help you understand how a program responds to unpredictable inputs.
Fuzzing has been around for a while. Defensics and Burp Suite are some examples in the traditional development world. There are also several web3/blockchain fuzzing tools available, such as Echidna and Foundry. However, Diligence Fuzzing is fuzzing as a service and makes everything a little simpler to implement. Which in the end means better audits and more secure contracts. So let’s look into it in more detail.
ConsenSys Diligence Fuzzing
Diligence Fuzzing (by ConsenSys, which is also behind ecosystem standards such as MetaMask and Infura) is a fuzzer built for web3 smart contracts. It:
Works from a formal spec that describes the expected behavior of your smart contract
Generates transaction sequences that might be able to violate your assertions
Uses advanced analysis to find inputs that cover a maximum amount of your smart contract code
Validates the business logic of the app and checks for functional correctness
Provides you with any findings
And all as a service with minimal work from you!
To use Diligence Fuzzing follow these three steps:
First, define your smart contract specs using Scribble.
Next, submit the code to Diligence to run your fuzzing.
Finally, with the audit report, fix and improve your code!
Fuzzing in Action
So let’s test it out and see it in action. We will use the Fuzzing CLI and Scribble to fuzz-test a sample smart contract.
Step 1: Sign up
First, sign up for access to the Diligence Fuzzing.
Step 2: Install dependencies
Next, install the Fuzzing CLI and Scribble. ConsenSys recommends that you have the latest versions of Node and Python. Be sure you are using at least Python 3.6 and Node 16. Then:
pip3 install diligence-fuzzing
npm i -g eth-scribble ganache truffle
Note: This requires a Linux, mac, or Linux subsystem with Windows. Windows Powershell has some complexities the team is working on. You can always use a github codespace (which creates a VScode-like-interface with a clean bootstrapped build) and install the above prerequisites via command line.
Step 3: Get an API key
Now you need to generate an API key to use the CLI. Visit the API Keys page and click on Create new API Key.
Step 4: Set up fuzzing configuration
Now we need a smart contract to fuzz! As part of their own tutorial, ConsenSys provides a sample smart contract to use. Let’s just use that one.
git clone https://github.com/ConsenSys/scribble-exercise-1.git
Open the .fuzz.yml file from the project and add in your API key for the “key” property at around line 25.
# .fuzz_token.yml
fuzz:
# Tell the CLI where to find the compiled contracts and compilation artifacts
build_directory: build/contracts
# The following address is going to be the main target for the fuzzing campaign
deployed_contract_address: "0xe78A0F7E598Cc8b0Bb87894B0F60dD2a88d6a8Ab"
# We'll do fuzzing with 2 cores 🚀
number_of_cores: 2
# Run the campaign for just 3 minutes.
time_limit: 3m
# Put the campaign in the Sribble Exercise 1 project
project: "Scribble Exercise 1"
# When the campaign is created it'll get a name <prefix>_<random_characters>
campaign_name_prefix: "ERC20 campaign"
# Point to your ganache node which holds the seed 🌱
rpc_url: "http://localhost:8545"
key: "INSERT YOUR API KEY HERE"
# This is the contract that the campaign will show coverage for/ map issues to etc
# It's a list of all the relevant contracts (don't worry about dependencies, we'll get those automatically 🙌)
targets:
- "contracts/vulnerableERC20.sol"
Note: Be sure to stop your fuzzing campaigns or set a time limit, or it might run for an unexpectedly long time. You’ll note from the above file that we set the time limit for our campaigns to three minutes.
Step 5: Define fuzzing properties
Notice also that we have our smart contract: contracts/vulnerableERC20.sol.
Next, we need to define the properties we want the fuzzer to check in the smart contract. We’ll use Scribble for this step. Scribble is a specification language that translates high-level specs into Solidity code. It allows you to annotate your contracts with properties and then transforms those annotations into concrete assertions that can be used by testing tools (such as Diligence Fuzzing). Pretty cool!
We will add the highlighted code segments to our contract:
pragma solidity ^0.6.0;
/// #invariant "balances are in sync"
unchecked_sum(_balances) == _totalSupply;
contract VulnerableToken {
This annotation will ensure that our total supply and balances are in sync.
Step 6: Run
Now we fuzz! Simply run this command:
make fuzz
Step 7: Evaluate results
After the fuzzer is done (it might take a minute or two to start up) we can get our results. We can either use the link the fuzzer gives us, or we can go to our dashboard.
Looking at properties, we can see what is being fuzzed and any violations. And guess what? We found a bug! Click on the line location button to see the offensive code.
For details, click Show transaction details. We can see the fuzzer called “transfer”:
Upon closer examination, we can now see what caused our bug.
The transfer_to and origin arguments are the same. There must be a security vulnerability when someone sends tokens to themselves. Let’s look in the source code to see what’s wrong.
function transfer(address _to, uint256 _value) external returns (bool) {
address from = msg.sender;
require(_value <= _balances[from]);
uint256 newBalanceFrom = _balances[from] - _value;
uint256 newBalanceTo = _balances[_to] + _value;
_balances[from] = newBalanceFrom;
_balances[_to] = newBalanceTo;
emit Transfer(msg.sender, _to, _value);
return true;
}
Yep! We can see that when the sender and recipient are the same, lines 30 and 31 will get a little weird—one is changing the value of the ‘from’ account, and one is changing the value of the ‘to’ account. The code assumes they are different accounts. But since they are the same account, by the time we get to line 31, the value we have is not the value we expect. It’s already been changed by the previous line.
We can fix this by adding the highlighted lines of code below:
function transfer(address _to, uint256 _value) external returns (bool) {
address from = msg.sender;
require(_value <= _balances[from]);
_balances[from] -= _value;
_balances[_to] += _value;
uint256 newBalanceFrom = _balances[from] - _value;
uint256 newBalanceTo = _balances[_to] + _value;
_balances[from] = newBalanceFrom;
_balances[_to] = newBalanceTo;
emit Transfer(msg.sender, _to, _value);
return true;
}
Here are several other technical details to be aware of:
The seed.js script does some setup work for you here. It deploys the contract to a test node. It can also do things like mint tokens, open pools, etc. It gives the fuzzer the right state to start.
The yml file has many config parameters that you can explore. Notably the contract address to fuzz, the API key, the time_limit for the fuzzing, and some others.
The CLI ships with an auto-config generator. Run fuzz generate-config to get some helpful Q&A for generating the config.
Smart Contract Audits — Use Fuzzing!
Fuzzing and Diligence Fuzzing-as-a-service is a powerful tool for testing auditing Ethereum blockchain smart contracts. Whether you are working in decentralized finance (DeFi), NFTs, or just starting in smart contract development, it can take you to the next level of identifying and fixing vulnerabilities in your smart contracts. Along with manual reviews, unit tests, manual testing, penetration testing, code reviews, and more, fuzzing should be a key part of your smart contract security audit process for a more secure and robust codebase.
Have a really great day!
