Latency And Infrastructure

What do you actually measure when you say “our feed handler is fast”? If you cannot point to a single end-to-end number tied to a specific trading-critical path, you will keep chasing improvements you cannot verify - and you will eventually ship changes that make fills worse while dashboards still look “healthy.”

Latency budgeting and measurement solve a specific problem: you turn a vague performance goal into a ledger of time spent in each hop of your system, then you validate that the measured end-to-end behavior matches the budget. After this chapter, you will be able to (1) define a latency budget for a chosen critical path, (2) pick metrics that match the decisions your trading system makes, and (3) verify end-to-end measurements with enough rigor to catch instrumentation mistakes.

You will also learn how to use the Latency Budget Ledger to separate “time we think we spend” from “time the market sees,” and how to run a tight measurement loop when you change code, NIC settings, or time synchronization.

Latency Budget Ledger: define the budget, pick the metrics, and lock the path

Start by choosing the trading-critical path you care about. For most firms, the path is not “from market to exchange” but “from market event arrival into your system to the moment your order reaches the exchange matching engine,” plus any decision latency inside your stack. The trap: teams often budget the wrong segment, like application-to-application time, while the real pain lives in kernel queues, serialization, or timestamping.

The Latency Budget Ledger forces you to write down every stage with a concrete unit (usually microseconds) and an explicit measurement method. Use it like a financial ledger: each line item has a measurable quantity and an uncertainty range. If you cannot measure a line item directly, you either add an instrumentation point or you mark it as “unresolved” and bound it using a controlled experiment.

A practical way to define the path: pick one order type and one decision trigger, then write the pipeline in the order you will instrument it. For example, for Nadia’s microstructure strategy (market microstructure quant, 34), the critical path might run from “best bid/offer update reaches the strategy process” to “order request bytes hit the exchange gateway,” because her signal updates must stay synchronized with micro-price changes. She does not budget “strategy compute time” in isolation; she budgets the full path that determines whether the order reflects the intended state.

Use these three rules when you write the ledger:

1. Pick timestamps you can control and interpret.

Decide which events produce timestamps: NIC transmit completion, kernel socket send timestamp, application “request built” timestamp, and exchange acknowledgment (or gateway-level receipt). Each timestamp must have a known meaning and known capture point. If you stamp “send” in application code but the OS queues the packet later, you have not measured what matters.

2. Budget in monotonic time, not wall-clock.

Use a monotonic clock for internal measurements (for example, `clock_gettime(CLOCK_MONOTONIC)`), and only use wall-clock for alignment with external systems. Monotonic time prevents jumps during NTP adjustments from poisoning your latency deltas.

3. Track both median and tail with the same path.

Don’t pick one without the other. Trading-critical behavior often fails in the tail due to queueing and contention. Record at least a central tendency (median) and a worst-case bound you actually observe (for example, 99th percentile, or “max over a run”), using the same instrumentation.

Once you have the path, choose metrics that map to execution outcomes. For low-latency systems, a good metric set usually includes: end-to-end latency for the critical path, application-to-kernel time (how long your process takes to hand off), kernel-to-wire time (how long the OS and NIC take to push bytes), and a timestamp-consistency check (does your “order built” timestamp align with the “packet sent” timestamp trend). If you only track one number, you will not know whether an improvement came from fewer queue waits or from your instrumentation drifting.

Finally, make the ledger executable. Each line item needs a measurement strategy: direct timestamp capture, packet trace correlation, or a controlled differential test. For direct capture, instrument at the producer and consumer of each stage. For correlation, use stable identifiers (sequence numbers in message payloads) so you can match one “order built” event to one “packet sent” event.

A ledger line item should read like: “Stage: strategy builds order → timestamp at order-build completion; Method: monotonic timestamp in strategy thread right before handoff; Expected: X µs; Uncertainty: Y µs; Validation: compare with packet trace and gateway logs.” If you cannot fill those fields, you do not yet have a usable budget.

...