In light of today’s SpaceX IPO, a story from inside Jane Street on the day of the 2012 Facebook IPO:
For weeks leading up to the May 18, 2012 IPO of Facebook, the entirety of Jane Street’s trading and tech teams had been preparing for a record trading day. Increased volumes, volatility, and dislocations were expected across equities stat arb, domestic ETFs, and index futures.
At the time, most of Jane Street’s US equities market data infrastructure ran through a third-party vendor’s middleware solution.
The market data product was essentially a box of FPGAs with firmware that could stream in raw data packets from exchanges like Nasdaq and convert them into normalized structs to be broadcast on our internal InfiniBand network.
It’s late morning of the Facebook IPO. Interest in the book was so enormous that the cross was delayed many hours. The entire trading floor was waiting in anticipation until, suddenly, all of Jane Street’s US equities market data went down.
The data vendor box had crashed. Chaos on the floor. In a panic and not knowing what else to do, we restarted the box. The market data from upstream exchanges was replayed and orderbooks were rebuilt, but the box just kept crashing, even after multiple restarts.
Why did this happen? It turns out that the box’s firmware had hardcoded the assumption that the struct field holding the number of orders on the orderbook level could be represented using 16 bits. Because surely, there couldn’t be more than 65,535 orders at a single level?
That assumption failed for the Facebook IPO. Overflow of a single byte caused Jane Street’s US trading operations to halt, which even then accounted for double-digit percentages of US equity/ETF volume. We ended up using a secondary data provider and narrowly averted catastrophe.
In trading there’s nothing better than meeting an unprecedented market event head-on and learning from it. SPCX traders, I hope there aren’t 2^24-1 orders at the support today.
Generated by Thread Navigator
Press ⌘ + S to quick-export