Markets at Machine Speed: The Eight Hops Between a Price Change and a Trade

Most descriptions of automated trading collapse the machinery into a single word. An "algorithm" sees a price and reacts. In practice, between the moment a matching engine changes a price and the moment an order arrives back at that engine, there is a chain of separate systems, each with a specific job: a feed handler that decodes the venue's data, a normalizer that turns it into a single internal picture of the market, a model that decides, a risk layer that checks the decision, an order builder, a gateway, and finally the venue's own machinery again. Each hop takes time, each can fail, and each fails in its own way.

This piece walks that chain one hop at a time. We describe what each stage does and why it exists, give a stylized budget for how long each takes in a well-built software pipeline, show where the microseconds go, and explain why the shape of the latency distribution matters more than its middle. We end with the failure mode that belongs to each hop and the control that catches it. The point is not that faster is better. It is that the chain is a system, and that it behaves well only when it is understood as one.

Eight Hops

Figure 1 lays out the chain. It begins inside the exchange, where the matching engine executes a trade or accepts a new order and, as a consequence, publishes an event to its market data feed. That event is the raw material for everything that follows. It leaves the venue's data gateway as a packet in the venue's own binary format, timestamped and sequence-numbered, and crosses a short length of fiber to a network card in the participant's cabinet. In the same building, that crossing takes on the order of a microsecond; across a city it takes hundreds of microseconds; across an ocean it takes tens of milliseconds, and the rest of this piece stops applying in any useful way.^[1]

The feed handler is the first stage the participant owns. Its job is narrow: read packets off the network card, check that the sequence numbers are contiguous, and decode each message from the venue's format into fields a program can use. It is narrow because it has to be. Every venue speaks a different protocol, the protocols change a few times a year, and a handler that tries to be clever is a handler that is wrong on the one message type it sees once a month. A good feed handler does exactly one thing and is tested against years of recorded packets.

Normalization is where the venues stop being different. A dozen feeds with a dozen conventions for representing a book update are turned into one internal picture of the order book: one convention for price and size, one meaning for a delete, one clock. This is the stage that lets the model be written once rather than once per venue, and it is also the stage that quietly absorbs the venues' quirks. Its output is a picture of the market that is, ideally, a few microseconds old.

The model is where the firm's ideas live and where most of the time is spent. It updates whatever features it maintains from the new book state, produces a forecast, compares that forecast with the current position and the cost of acting, and decides whether to do anything at all. In a pipeline built for speed the model is not a general-purpose program. It is a fixed sequence of arithmetic on data that is already in cache, with no allocation, no locks, and no branches whose outcome depends on anything but the numbers.^[2]

Risk stands between the decision and the wire. It checks the proposed order against position limits, notional limits, price collars around the last trade, message-rate limits, self-match rules, and the state of a kill switch that a person can throw. These checks are deliberately simple, because they have to be right when everything upstream is wrong. The order stage then encodes the checked decision into the venue's order protocol, stamps it with a unique identifier, and writes a record of it before it leaves. The gateway owns the session with the venue: sequence numbers, heartbeats, throttles, and the physical act of putting the message on the wire. From there the packet crosses the same short fiber in the opposite direction and enters the matching engine, which validates it, matches it or rests it on the book, and publishes the result, at which point the chain begins again.

Where the Round Trip Goes

The budgets printed in Figure 1 are stylized, and the point of them is their relative size rather than their absolute level. Figure 2 decomposes each hop's median time into three components: wire and serialization, which is the time spent moving bytes between a network card and memory or across fiber; processing, which is the arithmetic and logic the hop exists to do; and queueing and jitter, which is time spent waiting for something else, a core, a cache line, a lock, an interrupt, and which does no useful work at all.

Three things follow from the picture. First, the model is the largest single hop, but the five smaller software hops together take longer than the model does, and the wire crossings add more. A team that spends all of its effort on the model and none on the plumbing around it will find the round trip stubbornly slow. Second, the components are shortened by different means. Wire time is a matter of geography and hardware; processing time is a matter of code; queueing time is a matter of design, of keeping each stage on its own core with its data in its own cache and never asking it to wait on anything. Third, the queueing component is small at the median and is not small elsewhere. It is the part of every hop that grows when the market gets busy, and that is the subject of the next section.

The Tail Is the Product

A latency figure quoted as a single number is almost always the median, and the median is almost never what matters. The moments at which a pipeline is slow are the moments at which many messages arrive at once, and many messages arrive at once when something is happening: a data release, a large order, a repricing across a whole sector. Those are the moments the decision is worth the most. A pipeline that is fast on a quiet Tuesday afternoon and slow at the open has the wrong distribution, however good its median looks.

Figure 3 shows the distribution for a stylized version of the six hops the participant controls. Each hop's time is drawn from a lognormal distribution centered on the medians used in Figure 1, with a spread that is larger for the hops with more complex code paths; the bars show the 50th, 99th and 99.9th percentiles. The model hop has the widest spread, because it has the most branches and the most data to touch. The gateway is next, because it shares a network card with the outside world. The end-to-end bars deserve a closer look. The 99.9th percentile of the whole chain is well below the sum of the hops' 99.9th percentiles, because in this construction the hops' bad moments are independent and rarely coincide. In a real system they are not independent: the burst that slows the feed handler is the same burst that slows the model, and the end-to-end tail is correspondingly fatter than the figure suggests.^[3]

Two engineering consequences follow. Every hop must be timestamped, at the network card on the way in and on the way out and in software at each boundary in between, so that the tail can be attributed to the hop that produced it. And the number a team watches, alerts on and reports is the 99.9th percentile under load, not the mean. A change that improves the median and widens the tail is a regression, whatever the summary statistic says.

How Each Hop Fails

Speed is the visible property of the chain. Correctness is the property that determines whether the firm is still trading tomorrow. Each hop has a characteristic way of failing, and because the hops are narrow, the failures are specific enough that a specific control can be built to catch each one. Figure 4 pairs them. The pattern to notice is that most of the controls are invariants checked continuously rather than tests run occasionally: a book that is crossed, a feature that is not a number, an order whose price is far from the last trade, an acknowledgement that has not arrived. The pipeline checks its own sanity on every message, and stops when the check fails.

Two of the controls deserve comment because they are structural rather than local. The first is that the order object cannot reach the gateway without a risk stamp: the check is not a step that could be skipped but a property the gateway refuses to send without. The second is the kill switch, which is not a control on any one hop but on the chain as a whole. It cancels working orders, refuses new ones, and is wired to be operable by a person who does not need to understand what went wrong in order to stop it. Every other control in the table exists so that the kill switch is thrown as rarely as possible; the kill switch exists because the other controls will one day miss something.

How We Think About the Chain

At Oak St. the chain in Figure 1 is treated as one system with eight parts, not as eight systems that happen to be connected. That has concrete consequences for how the work is organized. Every hop has a time budget, and the budgets are reviewed together, so that a microsecond saved in the feed handler and a microsecond added in the gateway are recognized as the same trade. Every boundary is timestamped, so that the tail can be attributed. The controls in Figure 4 are part of the pipeline's design, built and tested alongside it, rather than a layer added after the fact. And the engineering effort goes where the distribution is worst, which is rarely where the median is worst.

It also has a consequence for how speed is valued. The chain is a constraint that has to be met for a strategy to work at all, in the same way that a clean point-in-time dataset is a constraint on research. Meeting it does not produce returns; failing to meet it forfeits them. The firms that do this well are not the ones with the fastest model. They are the ones that know, at every hop, how long it takes, how it fails, and what will notice when it does.