Is my ERC-4626 vault token up to the standard?

ERC-4626 is an extension of ERC-20 that introduces a standardized interface for tokenized vaults. Any ERC-4626 token also functions as an ERC-20 token, enabling the use of ERC-20 functions like transfer and approve. The primary purpose of ERC-4626 is to act as a vault, enabling users to deposit underlying assets (such as an ERC-20 token named A) in exchange for the vault's shares (also represented by an ERC-20 token, named S). Depositing assets into the vault allows users to yield interest, thereby offering the potential to withdraw a greater amount of token A than initially deposited, at a later time. To make a vault yield-bearing, it's essential for the quantity of token A within the vault to increase more rapidly than the production of token S. This objective can be accomplished by identifying optimal yield strategies through various approaches involving lending markets, aggregators, and other interest-bearing tokens.

Before the establishment of the ERC-4626 standard, there was no standardization for tokenized vaults. This absence hindered the integration and composability between various yield-bearing token protocols, resulting in complexity and time-consuming processes. Additionally, there was an elevated risk of vulnerabilities due to potential smart contract errors. With the introduction of the ERC-4626 standard by Joey Santoro and his team, developers of tokenized vaults now have a universal guideline to adhere to. This facilitates smoother and more secure integration across different protocols.

However, how confident can we be in the behavior of the contracts? Do they follow the standards as required? With the permissionless nature of blockchains, anyone can write and deploy smart contracts in blockchain ecosystems. Thus, it is important to check that they conform to the standard requirements as stated in their respective ERC standard. Without proper checks on deposit and/or withdraw assets functions, interacting with other contract protocols could lead to incompatibility and security issues. There are several ways to check if an ERC-4626 contract complies with its required standard. In this blog post, we will take a look at two different test suites, a16zcrypto’s ERC-4626 test suite and ERCx ¹ test suite. We will first briefly explain what we need to know about the ERC-4626 standard before discussing how to use both test suites to check for conformance of an ERC-4626 contract.

The ERC-4626 standard in a nutshell

As mentioned previously, an ERC-4626 contract is an extension of an ERC-20 contract. That is, an ERC-4626 tokenized vault must also implement all functions and events required as per ERC-20 standard. One exception is that an ERC-4626 tokenized vault can be non-transferrable, i.e., it may revert on calls to transfer or transferFrom. There are 2 main tokens managed by an ERC-4626 vault - asset (A) and share (S). The asset is the underlying ERC-20 token managed by the vault, which can be used to bear yield. The share is the ERC-20 token of the vault itself, which is used in exchange using any of the four main functions of ERC-4626 standard - deposit, mint, withdraw and redeem. Table 1 and Figure 1 summarize the effects of these functions.

*owner must either approve msg.sender or be the msg.sender in order to call the function

*owner must either approve msg.sender or be the msg.sender in order to call the function

Additionally, there are other important functions such as asset, totalAssets, convertTo {Assets, Shares}, preview {Deposit, Mint, Withdraw, Redeem} and max {Deposit, Mint, Withdraw, Redeem} , that allow querying for information about the vault, e.g., exchange ratio between A and S. Details of these functions and other requirements can be found on the ERC-4626 standard website.

Checking my ERC-4626 vault token

Every function mentioned in the previous section has strict requirements on its inputs, outputs and specification. As mentioned in the introduction, not conforming to the necessary requirements could lead to incompatibility and security issues for the vault token itself and any account or contract interacting with it. For example, having the wrong output type for deposit (uint256,address) to be uint8 instead of the required uint256 could lead to type issues when another contract is interacting with the vault token. Furthermore, if the token does not conform to the rounding requirements as stated in specified in the standard, any account could gain instant monetary gain due to rounding errors. For instance, if an account realizes that there is a loophole such that the amount of shares minted from depositing x amount of assets is greater than the amount of shares burnt from withdrawing the same x amount of assets, he/she could earn “free” shares in the vault token by repeatedly depositing and withdrawing the same amount of assets. Thus, it is important for any ERC-4626 vault token to conform to the required standard. But how can we check our vault tokens if they are up to requirement?

There are 2 test suites for ERC-4626 vault token: a16zcrypto’s ERC-4626 test suite and ERCx. Next, we briefly describe how to run each of the test suites and the properties they test before comparing them.

a16zcrypto

First, we note that a16zcrypto’s test suite can be run on both deployed and non-deployed contracts. For non-deployed contracts, using their source code, setting up the test suite is illustrated in Figure 2. The instructions of setting up the test suite, which can be found in the README file of the test suite repository, are as follows:

1. As the test suite runs forge test from Foundry, install Foundry and forge-std (a necessary dependency).
2. In the vault repository, create a test contract, say ERC-4626ConformanceTest.t.sol, in the test/ directory (create it if need be) and place the custom vault setup method ². An example is provided below (Figure 2):

Figure 2: Example of vault setup method for a16zcrypto’s test suite.

// SPDX-License-Identifier: AGPL-3.0
pragma solidity >=0.8.0 <0.9.0;

import "erc4626-tests/ERC4626.test.sol";

import { ERC20Mock } from "/path/to/mocks/ERC20Mock.sol";
import { ERC4626Mock } from "/path/to/mocks/ERC4626Mock.sol";

contract ERC4626StdTest is ERC4626Test {
    function setUp() public override {
        _underlying_ = address(new ERC20Mock("Mock ERC20", "MERC20", 18));
        _vault_ = address (new ERC4626Mock(ERC20Mock(_underlying_), "Mock ERC4626", "MERC4626")) ;
        _delta_ = 0;
        _vaultMayBeEmpty = false;
        _unlimitedAmount = false;
    }
}

3. Run forge test in console.

The generated output will state the passing and failing tests accordingly, with counter-examples for failing tests as seen in an example from running the test suite on Openzeppelin’s ERC4626Mock contract (Figure 3). To fully understand the output, please refer to the Foundry book.

Figure 3: Output generated from running a16zcrypto’s test suite on OZ’s ERC-4626Mock

To make the testing consistent between the two test suites, we ran both test suites on Openzeppelin’s ERC4626Mock contract. It took around 15 to 20 minutes to complete all the tests from a16zcrypto’s test suite. The reason for such a long run is that each test function is set up with a brand new vault with different sets of dummy users, balances and yields before testing the property the function is supposed to test. This means the vaults for the tests are independent of each other. As a result, the test suite provides better fuzzing coverage for each test and has a higher chance to catch edge cases if they exist.

The tested properties from the test suite includes:

• Round-trip properties: no one can make a free profit by depositing and immediately withdrawing back and forth.
• Functional correctness: the deposit, mint, withdraw, and redeem functions update the balance and allowance properly.
• The preview{Deposit,Redeem} functions MUST NOT over-estimate the exact amount.
• The preview{Mint,Withdraw} functions MUST NOT under-estimate the exact amount.
• The convertTo{Shares,Assets} functions “MUST NOT show any variations depending on the caller.”

The asset, totalAssets, and max{Deposit, Mint, Withdraw, Redeem} functions “MUST NOT revert.”

ERCx

Similar to a16zcrypto’s test suite, ERCx provides testing for both deployed and non-deployed contracts. However, an additional benefit of using ERCx is that it does not require any prior installation as everything can be run on the website directly. ERCx also provides an Open API for developers to have direct access to the test suite and services. For developers using VS Code for creating their ERC contracts, there is a Visual Studio Code (VS Code) plugin available that allows running the test suite with a click of a button. The instructions to run the test suite on the website are as follows:

1. (For deployed contracts) Copy and paste the address of the token you want to test in the text box which can be found on the left column of the Home page of ERCx website as seen in Figure 4. Next, choose the ERC standard test suite you want to test with (for this case, we will be running the ERC-4626 test suite) and the network (Mainnet, Sepolia or Goerlia) the address resides in. Finally, click the “TEST” button.

Figure 4: Screenshot of the ERCx website on how to test a deployed contract

2. (For non-deployed contracts) Copy and paste your source code contract in the text box which can be found on the right column of the Home page of ERCx website as seen in Figure 5 ³. Next, choose the ERC standard test suite you want to test with and the main contract class before clicking the “TEST” button.

Figure 5: Screenshot of the ERCx website on how to test a non-deployed contract

The summary of the test results (Figure 6) after running the ERCx test suite on Openzeppelin’s ERC4626Mock contract are structured into levels, where the descriptions of levels are as follows (Table 2):

Figure 6: Summary of the test result of Openzeppelin’s ERC4626Mock contract on ERCx website

The summary above provides the user a quick snapshot of how the contract has fared against the test suite in terms of conformance and desirable security properties. Any red failing test would immediately be noticeable to the user for future inspection. Figure 7 provides shows how the detailed test results look like:

Figure 7: Snippet of the test result of Openzeppelin’s ERC4626Mock contract on ERCx website

The report page presents a much more readable format of the results, which tells us the description of properties the contract passes or fails. The results can also be filtered via keywords through the “Filter the test results” text box as seen in Figure 7. The page also provides “Detailed Report” and “Test Logs” that may be beneficial to developers who are looking for a summarized report (similar to the report shown in Figure 3).

Running the test suite on Openzeppelin’s ERC4626Mock contract took less than 5 minutes. ERCx prioritizes test run speed and better coverage of tested properties over better fuzzing coverage. The main difference between both test suites is that the ERCx test suite fixes a single vault for all its tests instead of having different vaults for different tests like what a16zcrypto’s test suite does. The main reason is to have minimal modifications to the initial states so that it facilitates testing for both deployed and non-deployed contracts. The goal is to keep the current states in the EVM as they are, especially for deployed contracts, for testing, which is done so through a forked environment provided by Foundry. To provide different variations of scenarios, each test from the ERCx test suite will fuzz different shares’ and/or assets’ balance/s only if the test requires it. Although it limits the fuzz inputs for each test, it provides sufficient fuzz coverage for what each test is supposed to test. As a result, this greatly reduces the time taken to run the test suite.

The ERCx test suite provides an evaluation of a more comprehensive set of properties⁴. Beyond the properties tested by a16zcrypto’s test suite (i.e., round trip, functional correctness properties, etc) which ERCx also tests, there are many properties stated in the required ERC-4626 standard that are exclusively tested in the ERCx test suite. For example, regarding the maxDeposit(address) function, a16zcrypto’s test suite only checks to make sure that the function MUST not revert (via test_maxDeposit), whereas ERCx checks for that (via testMaxDepositNotRevert) and the following additional properties (Table 3):

Moreover, the ERCx test suite provides fingerprint tests that allows identifying the exact issues behind certain failed tests. For example, both a16zcrypto’s and ERCx’s test suite checks if the shares' balance of receiver indeed increases by the amount of shares output by a successful deposit(assets, receiver) call via the test functions test_deposit and testDepositIncreaseReceiverSharesAsExpected respectively. Other than this test, ERCx also provides the following two tests (Table 4) so that developers can identify the exact issue in the case where the testDepositIncreaseReceiverSharesAsExpected fails:

Comparison

The following table (Table 5) summarizes the comparison between the two ERC-4626 test suites, a16zcrypto’s and ERCx:

Table 5: Comparison between a16zcrypto’s and ERCx test suites

There are also certain properties stated in the ERC-4626 standard that cannot be tested through the use of Foundry. These properties will require a trained auditor/verification engineer in the blockchain ecosystem to check for correctness ⁵. For example, it is stated in the standard that the maxDeposit(address) “MUST factor in both global and user-specific limits, like if deposits are entirely disabled (even temporarily) it MUST return 0.” However, it is impossible to write a test function (at least in Foundry) to check for global limits or if deposits are entirely disabled.

Concluding Words

To conclude, if you want to test a deployed ERC-4626 contract (either on any of the testnet or on the mainnet itself), it is straightforward to test it with ERC xas all you need is an address. Testing it with a16zcrypto’s test suite seems possible even though when we run it on deployed contracts the execution was not finished after one hour. For testing of non-deployed ERC-4626 contracts, running a16zcrypto’s and ERCx’s test suites is highly recommended. On one hand, the former provides better fuzzing coverage for each property test, and, hence, has a higher chance to catch edge cases if they exist. On the other hand, the latter provides better coverage of properties required from the ERC-4626 standard and can be run in a much shorter time. It is also important to note that both test suites run properties tests through fuzzing. Hence, the results may contain false negatives, i.e., some tests that are supposed to fail may pass because the fuzzing test cannot find counterexamples within the limited number of runs. To guarantee that a contract holds certain properties, one must use formal verification tools such as Kontrol.

Getting Further and Contributing

Like our ERC-4626 test suite? Want more features for our ERCx tool? Let us know! We are actively looking for feedback on the tool. So if you have any suggestions or features that would improve how you integrate ERCx into your workflow, please contact us. You can also find us at Twitter, Discord and Telegram for more information and updates.

Footnotes

1. You can also check out our previous blog post, titled “Introducing ERCx: Conformance and Property-checking for ERC Tokens”, where we first introduced ERCx. ↩

2. Note that the test suite allows some customizations such as _delta, which is the maximum approximation error. Please refer to the test suite repository to find out more. ↩

3. Note that there are some minor requirements such as the contract has to be flatten (i.e., no imports required) and the constructor of the main contract class does not require any input argument, which can be found in the Developers page. ↩

4.  Please refer to this page to see the full list of properties tests from the ERCx’s ERC-4626 test suite. ↩

5.  If you need a trained auditor/verification engineer to check and verify your contract, you can contact us on our website. You can also find us at Twitter, Discord and Telegram. ↩