GM Fox Fam,
I have completed the final incident report for the event that occurred on February 1, wherein the DFC instructed signers to execute the sale of 99k RUNE. This decision was made during a critical moment for RUNE, as the price fell dramatically from a previous stable average of around $3.35 to nearly $1.29, and eventually near parity at $1.00 during the transaction execution. This significant drop occurred in conjunction with the departure of Thorchain’s first node operators and many contributors, which precipitated an urgent need for action. This report details the sequence of those events and identifies several critical lapses in our processes, particularly in the execution and oversight by former signers, which intensified the financial impact.
Incident Response Journal
Incident Details
• Date & Time of Incident: February 1, 2025, 11:00 AM – 8:00 PM PST
• Incident ID: rune-1
• Reported By: ProfMcCarthy, Tokenomics WS Leader
• tokenomics@shapeshiftdao.org
• Affected Token/Chain/Pool/etc: Thorchain asset (RUNE) – no pool involvement
Who
• Individuals Involved:
@PTT, @Prof, @CJ, @Bishop, @giantkin (giantkin@giantkin.com), @Tim Black
• Internal Teams Engaged:
DFC (Decentralized Finance Committee)
• External Entities Notified:
n/a
What
• Description of Incident:
A flash crash in the RUNE token occurred due to a combination of factors including the unbonding of Thorchain node operators, departure of key project participants, and a precipitous price drop. This triggered a firesale event, with the team rapidly discussing and executing buyback strategies, adjusting for market volatility and liquidity challenges.
• Type of Incident:
Flash crash (specific to one asset)
• Severity Level:
Critical
• Root Cause Analysis:
• Unbonding of Thorchain node operators
• Departure of several key players from the Thorchain project
• Precipitous price drop triggering panic selling
• Inadequate buy-side liquidity during extreme sell pressure
Where
• Location of Incident:
Thorchain token (RUNE)
• Affected Systems & Infrastructure:
Thorchain Multisig
When
• Time of Initial Detection:
Approximately 1:44 PM PST (first internal alert noted)
• Time of Response Initiation:
1:44 PM PST, as team members began active monitoring and rapid discussions
• Time of Containment:
Ongoing adjustments through incremental buybacks until the hard stop at 2:00 PM PST
• Time of Resolution:
7 PM PST
• Duration of Incident:
Approximately 7 hours (with extended monitoring and analysis in the subsequent days)
Why
• Reason for Incident:
The flash crash was triggered by market instability caused by significant unbonding activity and the departure of key figures, which led to a cascade of sell orders and rapid price deterioration.
• Vulnerabilities Exploited:
• Concentrated liquidity issues on the buy side
• Heavy reliance on the commitment of key project participants
• Limited crisis communication protocols under high-pressure situations
• Contributing Factors:
• Rapid human response under stress
• Technical vulnerabilities in liquidity and transaction execution
• External market pressures and negative sentiment
Incident Response Actions
| Step No. | Action Taken | Responsible Party | Timestamp | Status |
|---|---|---|---|---|
| 1 | Detection & Identification: Initial alert triggered by rapid price drop and internal observations. | Monitoring Team (DFC) | 1:44 PM PST | Completed |
| 2 | Containment: Immediate discussions on sale, buyback strategy; and the execution of incremental transactions | @Prof, @GiantKin, @CJTCK | 1:44 PM – 2:00 PM PST | Completed |
| 3 | Eradication/Mitigation: Dynamic adjustment of buyback percentages based on market conditions; monitoring of node unstaking events overnight. | @Prof, @GiantKin, @PTT | 1:45 PM – 7:31 PM PST | Completed/Ongoing |
| 4 | Recovery: Ongoing post-event market stabilization analysis and follow-up communications to confirm transaction execution. | Entire Incident Response Team (DFC) | 2:00 PM PST-7:23pm PST | Completed |
| 5 | Lessons Learned (Post Mortem): Comprehensive review of communication logs, transaction timings, and strategy adjustments; assignment of post-event action items. | @Prof, @CJ | Initiated February 4, 2025 | In Progress |
Additional Communication & Transaction Execution Timeline
• Signer Availability and Task Confirmation:
• ProfMcCarthy requested confirmation from signers regarding availability to execute a queued Thorchain RUNE sale.
• Signers confirmed availability with language emphasizing that once a decision was made, instructions would be promptly executed.
• Key Language Used:
• “When you decide let the signers know and we’ll get it done.”
• “There’s no use asking for responses now if we’re not doing anything now. Give the task when you have a task to give. Thx.”
• Transaction Execution Timing:
• Initial instruction for execution was sent at 1:00 PM.
• Final vote confirmation was sent at 1:43 PM.
RUNE price was 1.29 at the time of the vote
• A delay occurred when a signer (Signer 3) responded at 2:22 PM regarding an error in the vote approved by the DFC (USDC being the operational address instead of the main ETH address).
• The error was corrected within 7 minutes at 2:29 PM.
• Despite the correction, final execution was delayed until 6:23 PM when @giantkin noted “it might be too late.”
• At 6:27 PM, a follow-up confirmed that instructions were reissued to a signer.
• Final confirmation came at 7:23 PM, with the signer stating, “yep it was executed today.”
https://viewblock.io/thorchain/tx/57CB36EE13A18C21C9689A064C554A2EA903DA8783C5C4EDCBFD46100DF44FC9
At the time of sale, the price of RUNE was 1 USDC = 0.98283835 RUNE
Supporting Documentation
• Logs & Screenshots:
Detailed internal communication logs capturing both the firesale event and post-event signer coordination (February 1 and February 4, 2025).
• Correspondences:
Emails and messaging transcripts from key DFC members, including specific timestamps for instruction issuance, error handling, and transaction finalization.
• Incident Timeline:
| Step No. | Action Taken | Responsible Party | Timestamp | Status |
|---|---|---|---|---|
| 1 | Initial detection & alert | Monitoring Team (DFC) | 1:44 PM PST, Feb 1 | Completed |
| 2 | Buyback strategy discussion & execution | @GiantKin, @Prof, @CJTCK | 1:44 PM – 2:00 PM PST | Completed |
| 3 | Ongoing market monitoring and dynamic adjustments | @Prof, @GiantKin, @PTT | 1:45 PM – 7:31 PM PST | Completed/Ongoing |
| 4 | Transaction instruction and signer coordination | @Prof, Signers | 1:00 PM – 7:23 PM, Feb 4 | Completed |
| 5 | Post mortem planning and assignment of responsibilities | @Prof, @CJ | February 4, 2025 | In Progress |
Post-Mortem Reflection (RBT Framework)
Roses
- Quick DFC mobilization
- Responsive collaboration under stress
Thorns
- Signer miscommunication
- Lack of weekend/off-hour structure
- Lack of range for executing the trade
Buds
- Formalize signer instructions & checklists
- Create weekend crisis command structure
- Automate volatility alerts
- More consistent monitoring/adjusting volatile treasury assets (non-fox, non-stables, non-btc, non-eth)
Follow-Up Action Items
Emergency & Fire Drills
- Emergency Driver: @Tim_FBL
Monitoring & Early Warnings
- EWS Signals: @Tim_FBL w/ @PTT, @Prof
- how to automate this with messages or alerts?
- Ongoing Market Analysis: @Prof
Risk Management
- Off-Hours Chain of Command: @CJ
- Stablecoin Depeg Protocol: @DFC
- Risk Framework Maintenance: @CJ (monthly)
Documentation & Communication
- Post-Mortem Journaling: @Prof
- Ops Sprint Comms Flow: @Hpayne (proposal)
Portfolio Strategy
- Monthly Simulation given different market conditions: @Prof
Final Details
- Submitted By: ProfMcCarthy
- Reviewed By: DFC
- Submission Date: April 1
- Approval Date: April 3rd