In modern markets, speed kills signal.
High-frequency trading (HFT) has flooded the limit order book (LOB) with millisecond-scale activity: orders that flash in and out without intention to execute. These “flickering quotes”—the strategic residue of market makers, latency arbitrageurs, and spoofers—inject enormous noise into directional indicators like Order Book Imbalance (OBI). For firms trying to build real-time trading signals, the result is a muddied picture: imbalance measures that correlate weakly with returns, and worse, mislead causally.
A new paper by Anantha, Jain, and Maiti presents a compelling remedy: structural filtration. By filtering the LOB in real time based on observable features—such as order lifetime, update frequency, and inter-update delay—they dramatically enhance the clarity of imbalance signals. But the most striking insight? Filtering alone isn’t enough unless you’re looking in the right place: the trades.
The Setup: Filter First, Diagnose Later
The authors construct a three-layer framework to evaluate signal quality after filtration:
| Layer | Diagnostic | Measures |
|---|---|---|
| 1 | Pearson Correlation | Raw co-movement between OBI and short-term returns |
| 2 | Regime Alignment | Structural alignment of binned OBI and returns |
| 3 | Hawkes Excitation | Causal flow from OBI regimes to return regimes |
They apply this to tick-level data on BANKNIFTY futures, comparing three real-time filters:
- Lifetime Filter (LF) – Discard orders alive for < threshold (e.g., 500ms)
- Modification Count Filter (MF) – Discard orders with too many updates
- Modification Time Filter (MTF) – Discard orders with rapid-fire modifications before exit
All imbalance signals are computed over a 10-second lookback window and evaluated against returns over a 1-second forward window.
Results: MTF Wins the Correlation Battle
The Modification-Time Filter (MTF) consistently improved both raw correlation and regime alignment metrics:
- Correlation: +11% over unfiltered baseline
- Cross-regime structure: lower error in matching OBI/return states
Interestingly, Modification Count (MF) often degraded performance—possibly because some frequently updated orders still contain informative intent (e.g., algorithmic quote shading).
But here’s the catch: Causal strength—measured via Hawkes process norms—barely improved under any filter. The filtered OBI may align better statistically, but it doesn’t move prices in a causal sense.
Enter OBI(T): Trades as Ground Truth
To resolve this paradox, the authors pivot to a new definition: OBI(T)—imbalance computed purely from trade executions (signed volume difference between buys and sells).
This version is:
- Immune to flickering quotes
- Anchored in realized commitment, not transient intent
When they reapply the same filters to OBI(T):
- Hawkes excitation norms jump dramatically
- MTF boosts causal excitation score by +100% on active trading days
Implication: If you’re trying to model or forecast short-term returns using imbalance, it’s not enough to clean the LOB—you need to anchor in trades.
Broader Takeaways
-
Associative vs. Causal Alignment: Filtering helps with correlation, but causality requires signals that reflect executed conviction. Flickering quotes are noise not just statistically, but causally.
-
Trade-Based Signals Matter: For strategy design—especially if it depends on triggering alpha via order flow—trade-based imbalance measures (like OBI(T)) filtered in real-time offer far more reliable ground truth.
-
Filtering Is Real-Time Viable: All filters used rely on information observable at the time (e.g., order age, mod count), not future hindsight. This makes them implementable in live systems.
-
Don’t Just Filter—Diagnose: The authors’ diagnostic ladder (correlation → regime → excitation) provides a powerful tool for firms auditing the informational content of their features, not just their backtest performance.
Final Word: Design Signals, Not Just Features
In high-frequency trading, it’s tempting to compute more, faster. But this paper reminds us: what you measure is only as good as how you clean it.
Structural filtration gives you sharper edges—but if you’re still computing imbalance from quotes rather than trades, you’re painting with fog. To move beyond association and toward true signal causality, build your indicators on what traders do, not just what they pretend.
Cognaptus: Automate the Present, Incubate the Future.