A case study of how a web3 incident response should operate
The Highland Security team looks at the November 2023 Aave incident as a case study of an effective incident response operation in web3. If you want more more information an approach for your project, contact us and lets talk security! If you like this post subscribe to our blog
Over the course of 2023, we have witnessed 1B+ in losses to hacking activity. With each successful hack, the community also got to witness a wide array of protocol teams’ responses to the incident and governance crisis management apparatuses in action. A recent incident involving the Aave protocol serves as a potent case study on what the web3 community should expect from protocol security programs moving forward to continue to raise the security bar and protect their users. On November 4, 2023, Aave, a widely-used DeFi protocol, notified their community that a critical vulnerability was reported by a security researcher through their bug bounty program. In this post our team will take a look at the core components of Aave’s public response to the incident and their security program in action.
What happened?
The Aave protocol, known for its innovation in the DeFi lending space, received a high risk vulnerability report affecting both versions 2 and 3 of the protocol from their Bug Bounty program. According to their governance forum update the report was reviewed further by the team and ultimately updated to a critical risk requiring immediate action to protect customer funds. At present time, little is known about the details of the vulnerability, only that it posed a significant enough threat to take emergency action and to mitigate the risk the Aave Guardians disabled the stable rate mode borrowing feature of Aave which was possibly vulnerable. The immediate and strategic responses that followed set the stage for a good example of what we think is a good example of a top-notch incident response.
Immediate and Coordinated Security Action
One of the key takeaways from the Aave incident is the speed and coordination with which the Aave Community Guardian, comprised of community-elected experts from organizations such as BGD Labs and Governance House DAO, responded to the vulnerability. This swift reaction is a testament to the effectiveness of having a community guardian structure in place and a plan of action established ahead of time that can be tested.
The response included pausing the Aave V2 Ethereum Market and freezing specific assets on Aave V2 and V3 across various blockchains like Ethereum, Optimism, Arbitrum, Avalanche, and Polygon. These actions ensured that users could still withdraw and repay from frozen assets, even though new borrowing or supplying activities were temporarily suspended. Meanwhile, markets not impacted by the vulnerability continued to operate without interruption. The mechanisms of pausing/upgrading deployed smart contracts is a controversial topic in decentralization circles, however given the significant risks posed by threats today some security actions are a necessity for any serious project. The Aave situation shows that decentralization principles can coexist with protocol security actions if architected properly.
Transparency
In the wake of the vulnerability discovery, Aave's response was not limited to halting operations. A governance proposal was introduced, outlining the intention to disable the stable borrow rate for all assets across all pools on all networks. This action, while temporarily affecting certain assets, is aimed at providing a permanent resolution to the reported vulnerability.
The proposal also detailed specific technical calls for adjustments within the Aave protocol across different blockchain networks, including disabling stable borrowing rates and unfreezing assets that were frozen as a precautionary measure. This comprehensive approach reflects the importance of thoroughly addressing and rectifying vulnerabilities. In addition, a subsequent update was posted on Nov 6 as the situation evolved with updates on the response actions taken and next steps for transparency of response actions. In this case the Aave team looks to be implementing a new oracle feature for a Liquidation Grace period after the protocol is unpaused. Having the right infrastructure and process in place beforehand goes a long way to assuring the community that their funds are safe in times of crisis.
Implications on Broader DeFi Ecosystem
Aave is so ingrained into the fiber of the DeFI ecosystem, any vulnerability like this tends to have ripple effects l for other projects that had forked Aave's codebase. This is the dangerous underbelly of forking and extending a protocol. To their credit, Aave had a plan for this aspect also. In their latest update on Nov 6th, Aave Guardians indicated that they had reached out to affected V2/V3 forks of their protocol with information to protect their communities. This proactive response illustrates the interconnectedness of the DeFi ecosystem and how a vulnerability in one platform can have cascading implications. It also underscores the importance of collaboration among DeFi projects to collectively strengthen security measures.
This goes above and beyond what we typically see in the web3 ecosystem today. This type of action and coordinated response to help other affected communities is what's truly needed for software security in web3 to mature substantially.Â
Next steps for Aave
At time of writing, the affected Aave pools are still paused however, the latest update by the Aave guardians have drafted a governance proposal which addresses the issue by disabling the possibly vulnerable stable debt token functionality. There is a keen anticipation for a detailed postmortem of the incident and some community members were not satisfied with disabling stable debt. . This postmortem is expected to provide a thorough analysis of the vulnerability and the steps taken from its discovery to resolution. It is this transparency and willingness to be held accountable that exemplify the best practices of a top-notch web3 security program. For further updates on the Aave pools you can get updates from the Aave social media, the community updates thread, and governance fix proposal
Lessons Learned
As we await the post mortem from the Aave team, below are some takeaways from this case study that you can implement into your web3 security program:Â
Have a designated communication protocol for vulnerability reports: Aave was able to take action to protect community funds because a white hat security report through their bug bounty program. Every launched protocol should implement a dedicated and publicly promoted way to communicate potential vulnerabilities. It could be as simple as an email address and web page or more intricate like a paid program. It doesn't really matter as long as it's monitored and reports taken seriously.
Create a response plan that can be tested and improved upon: Aave protocol had the foresight to implement security mechanisms and associated transparency with guardian processes to enable them to quickly execute their response plan. This goes a long way to mitigating impact and quickly recovering from an incident. Every launched protocol should consider making a plan on how to respond and transparently share the key elements of the plan with their community.
Periodically test your plan and adapt to threats: Testing if your response plan is feasible and able to be implemented before an incident happens is just as important as having one. Otherwise it's just a piece of paper. While Aave has used their security features to respond to past threats, it appears that in addition to fixing the bug, the Liquidations functionality will also have to be updated to handle the condition after a pool is unpaused, requiring additional work on top of any bug fixes.Â
If you are interested in improving your web3 security program with some of these strategies, contact us today for a free consultation!
Comments