Low latency · FPGA · Risk

A Pre-Trade Risk Gate in One FPGA Clock Cycle, and What I Would Fix Before Production

How the SEC 15c3-5 credit, fat-finger, and duplicate-order checks in my FPGA market-making thesis run in parallel in a single 3.1 ns cycle, and a candid review of the design: overflow, state that never resets, timing, and hardware and software limits that may disagree by 100x.

Shreejit Verma · · 7 min read

In Trishul, my market-making thesis system, the pre-trade risk check is not a function the strategy calls. It is a hardware stage wired in series between the decision logic and the order encoder, so an order that fails a check never reaches the wire. All three checks run in parallel and the verdict is registered in one clock cycle, about 3.1 ns.

Fitting the checks into one cycle turned out to be the easy part. Rereading the module months later, the hard problems were at the edges: integer widths, state that only ever grows, and two implementations of the same rules that quietly disagree. This post walks through the design, then reviews it the way I would review someone else's pull request.

What the rule asks for

SEC Rule 15c3-5, the Market Access Rule, requires a broker-dealer with market access to run risk controls before orders reach an exchange. The financial controls in paragraph (c)(1) must:

Paragraph (d) adds that the controls must be under the firm's direct and exclusive control. The rule says nothing about latency. That is the engineering problem: a check that adds microseconds to every order is a cost a market maker pays on every quote.

Why the check lives on the FPGA

Trishul keeps the whole tick-to-order path on the FPGA: it parses NASDAQ ITCH 5.0, maintains the L2 book, runs a fixed-point policy network, and encodes OUCH 5.0 orders. In simulation that path has a median of 822 ns and a 99th percentile of 1,180 ns. A round trip to the CPU for a risk decision would cost more than the rest of the path combined.

Putting the gate in hardware also changes what "mandatory" means. In software, a risk check is a call that a code path can skip. In the FPGA, the gate is the only wire between the decision core and the encoder, so there is no path around it.

The design: three checks, one cycle

The module, risk_gate.v, takes an order (price, quantity, and a 64-bit hash) from the decision core and either forwards it to the encoder on the next clock edge or drops it and raises an alarm. Each check is combinational logic evaluated in parallel:

wire [PRICE_W+QTY_W-1:0] order_notional = in_order_price * in_order_qty;

wire check_credit     = (total_exposure + order_notional[PRICE_W-1:0] <= LIMIT_NOTIONAL);
wire check_fat_finger = (in_order_price <= cfg_max_price) &&
                        (in_order_price >= cfg_min_price) &&
                        (in_order_qty   <= cfg_max_qty);
wire check_duplicate  = (in_order_hash != last_order_hash) ||
                        (timestamp_counter > 32'd322); // 1 us at 322 MHz

A clocked block ANDs the three results. On a pass it registers the order to the output and updates the exposure, hash, and window counter. On a fail it drops the order, sets the alarm flag for each failed check, and clears the flags of checks that passed. Because the three checks are independent, adding a fourth costs logic area, not latency.

The C++ control plane carries a software twin, PretradeChecker, for the backtest and simulation path. It applies the same three checks plus a position limit, so the strategy can be tested end to end without hardware.

Reviewing my own design

Here is what I would flag if this landed in my review queue, most serious first. Each item is in the public repository, so you can check it against the RTL.

1. The hardware and software limits can disagree by 100x

The RTL default is LIMIT_NOTIONAL = 64'd1000000000, commented as $10 million. The C++ side stores prices with PRICE_SCALE = 10000. If the FPGA uses the same scale, as it would for prices to cross the PCIe boundary unchanged, that constant is $100,000. The C++ checker enforces $10 million. Both paths pass their own tests, and a backtest would never reveal that production enforces a limit one hundred times tighter.

Fix: generate both constants from one definition, and run the same order vectors through the C++ checker and the RTL in simulation, failing the build when their verdicts differ.

2. Notional can overflow and pass the credit check

The product of a 64-bit price and a 32-bit quantity is 96 bits, but the credit check uses only the low 64. The sum with total_exposure is also 64 bits, so it can wrap. A large enough order can wrap to a small number and pass. The fat-finger caps make this unlikely with sane thresholds, but a risk control should not depend on another control being configured correctly.

Fix: compare at full width, or saturate, so an overflow always rejects.

3. Exposure only ever goes up

total_exposure increases on every accepted order and never decreases. There is no input for fills or cancels, and no way to reset exposure short of a full module reset (rst_n). That makes it a session ceiling on gross notional sent, not a credit control. The software twin has the matching gap: its execution callback is a stub, and its position counter is never updated, so the position limit only ever compares a single order against zero.

Fix: feed cancel and fill reports back into both implementations, and track open exposure and position, not just orders sent.

4. One cycle at 322 MHz is optimistic for a 64 by 32 multiply

The notional multiply, a 64-bit add, and the compare all sit in one 3.1 ns cycle. On real FPGA fabric that multiply spans several cascaded DSP slices, and I expect it to fail timing at this clock. I have not run it through synthesis and timing analysis, so treat this as a prediction. The fix is cheap: compute the notional in a first pipeline stage while the fat-finger checks run, then compare in the second. That costs one cycle, about 3.1 ns, still a rounding error against the rest of the tick-to-order path.

5. Smaller items

What I take from it

The single-cycle number is the least interesting thing about this module. Parallel checks are easy to make fast. What decides whether a risk control works is what happens at the boundaries: the width of every intermediate value, every piece of state and the event that resets it, and every place the same rule exists twice. Those are the things I would test first, ahead of latency.

Trishul is research software: it runs in simulation and RTL testbenches, not on an exchange connection. The items above are what I would fix before treating its risk gate as anything more than a design study.

References