The Order Book Revisited

I have been thinking more about a market order book. It is an interesting challenge for a number of reasons:

• It is a real-world scaling problem
• The logic seems simple but nuances add complexity
• It is something that can be done completely with commodity hardware, but is big enough to be of interest to hardware designers and specialty hardware makers.

Using an iterative and incremental approach, let’s build an order book. I’m not going to talk about what a market order book is here. If you don’t know, ask Google.

• Phase 1 will be completely in-memory. We will push transactions as fast as possible and explore performance. We will get some baseline numbers for adds, updates, deletes, and queries. The performance numbers will shape where we will look for improvements.
• Phase 2 will implement a few of those “improvements” to see how they affect the baseline numbers.
• Phase 3 will introduce networking and reliability. We will examine TCP and UDP Multicast variants, and see how design decisions limit scalability and determinism.
• Phase 4 will look at specialty hardware and the tradeoffs in complexity and efficiency.

Phases 1 and 2 are certainly within my wheelhouse. This is something I’ve done in the last go-round. I hope to do it again with a little more attention to detail.

Phases 3 and 4 are ambitious. While I have experience in all 4 phases, I believe I will be learning a great deal in these phases. Let’s get started…

(A note to my future self: Come back to this post and link the previous posts as well as the future phase discussions)

Here is Phase 1