Hong Kong-based cryptocurrency exchange BitMex has been at the center of attention since Thursday, March 12, when the Bitcoin price crashed from $7,900 to as low as $3,600, recording its worst single-day drop in seven years.
Data from crypto data outfits showed that during Bitcoin's sell-off this evening, BitMEX liquidated $1.2 billion worth of long contracts on its platform alone, causing one of the most intense long squeezes in the crypto history.
And while reports of whales dumping on BitMEX explained the downward spiral of the Bitcoin market, crypto investors weren't convinced that figures do not say anything about deliberate manipulation. A few hours later, BitMEX confirmed it had come under an aggressive DDoS Attack when it went offline for around 25 minutes, during which the Bitcoin price started to recover across other exchanges.
Today, BitMEX's chief technical officer Samuel Reed further revealed in a series of tweets that the exchange experienced 'a botnet attack' exposing flaws in its AWS servers. Reed added that botnet owners had access to "an endpoint that was consistently, reliably slow" and thus were able to carry out malicious activities on March 13 at both 02:15 UTC and 12:56 UTC.
BitMEX's CTO added that hackers have been probing the system for some time and were also responsible for an attack that took place last month. February's attack, however, was absorbed by BitMEX's normal DDoS mitigation strategies.
He further explains:
"March 13 was a change in strategy for them. The botnet found an endpoint that was consistently, reliably slow. The query they hit did a 400ms reverse sequential scan rather than using the index (Parallel Index Scan / Gather Merge for PG fans) because an ANALYZE hadn't been automatically run for too long by RDS defaults."
He continued, "Thousands of those scans in parallel caused the database to start swapping, pegged to 100% CPU, with over 99% of that as iowait. On AWS, this looks quite a bit like a dying EBS volume, so we failed over the database and service resumed."
Users blame BitMEX for relying too much on third-party servers
In the incident's aftermath, Reed said the exchange was focused on rebuilding and recovering its system, having already identified the slow query and fixed it.
"After the second attack, we. We're making systemic changes on our backend to ensure this can't happen again, and re-reviewing older systems to simplify, de-couple, isolate, and improve perf," he added.
In essence, BitMEX is saying that a malicious party may have been able to manipulate its AWS servers, noting that the hackers were advanced, persistent, and patient and had waited to collect a significant volume of data before executing the attack.
But against a backdrop of heavy losses for traders, BitMEX is currently fielding intense criticism. Aside from technical reliability, one theory even suggests the exchange can't simply blame a "hardware issue" for its outage. Albeit traders are protected by the insurance fund, it is still worrisome that a giant derivative platform such as Bitmex has been relying on third-party infrastructure.
Haha they are renting VMs. They are sharing servers with god knows who else. No way to verify security. AWS is for test/dev. Not large algorithmic trading platforms. No wonder they have so many "Order submission errors"
— Schrödinger (@PoliteSociety3) March 16, 2020
.
