Binance CEO Changpeng Zhao has a simple way to describe his mental state after the massive hack on his exchange earlier this month.
“I am not gonna deny it. My first reaction was: ‘Fuck!’, the second and third reactions were also the same. A few moments after that, I began to come to terms with it, ‘Well that sucks! What do we do now? Lots of people are waiting for me, some for instructions, some for information and some for reassurance. Lots to do, let’s just get on with it…
After 10 seconds of the ‘Fuck, Fuck, Fuck’ state, I did a quick mental calculation. 7000 BTC, fine, I know we have more than that in our own BTC funds alone. There is enough. A second calculation eased my mind, this was about the same as a quarterly burn we did about a year ago, not such a big deal.”
In a new blog post, Zhao recounts the behind-the-scenes story of how the Binance team handled the past two weeks after hackers stole 7,000 Bitcoin worth $40 million from one of the exchange’s hot wallets.
He also touches on the thought process behind his much maligned and abandoned “reorg” idea, which would have involved attempting to incentivize 51% of miners that power the Bitcoin network to essentially reverse and reorganize recent Bitcoin transactions, including the massive Binance hack.
After the attack, Zhao scheduled an ask-me-anything session with the public.
“Before the AMA, I had been up all night and I was really feeling the effects. So, I took a 15-minute nap just before the AMA. Upon waking up, my team told me there was an interesting proposal from a Bitcoin Core developer. I read it for a few seconds. It involved something called a ‘reorg’. While I know it’s technically possible for a rollback in a 51% attack scenario, it never occurred to me that it is also technically possible to change one transaction and keep all other transactions intact, while hugely incentivizing the miners. The discussion was already pretty hot on Twitter, so I mentioned it in the AMA as something that was suggested. Little did I know, it was a taboo topic. Lesson learned.”
Zhao quickly ditched the idea, even going so far as to say it was “not possible.” After the hack Zhao notes that the hackers gained control of several user accounts and made large withdrawal requests that bypassed Binance’s pre-withdrawal risk management checks.
“Our post-withdrawal risk monitoring system caught it immediately and suspended all subsequent withdrawals. While things are crystal clear in hindsight, at that moment, we weren’t 100% sure what exactly happened. Was it an actual user action? A glitch in the system? Or maybe a hack? As we were still evaluating the situation at the time, we decided to proceed with caution.”
Zhao then tweeted that Binance’s withdrawal servers were in unscheduled maintenance mode, and the exchange’s team continued to investigate what happened. After they were relatively certain only one transaction was impacted, they decided to notify the public about the security incident, estimating that withdrawals would be suspended for one week.
“In the world of technology, you can never accurately estimate how long changes might take. It is quite different when you compare it to repeated, predictable work. Regardless, our users and community needed an estimate, and once communicated, it became a target deadline for our team to deliver. I did not know how the community would react to a one-week withdrawal suspension, but luckily, being transparent paid off and we received tremendous support from our amazing community.
Lesson: During a crisis, constant and transparent communication is key.”
Zhao says his team went into “war mode,” implementing new security measures to further ring-fence the system.
“Our team pushed on, day and night. In places where we congregate in small temp ‘offices,’ we had temp beds from Ikea rolled out. I won’t go into the details here, as we don’t disclose our security practices, but to bring the system back online within one week, all of our teams did more than a quarter’s worth of work in that one week.”
Going forward, the CEO says the hack could actually be “a blessing in disguise.”
“Security is a never-ending practice; there are always improvements to be made. We have implemented many of them in this last week and will continue to implement more in the future. Given this incident, Binance has actually become far more secure than before, not just in the affected areas, but as a whole.”
[the_ad id="42537"] [the_ad id="42536"]