When financial AI meets the optimizer arms race, the stakes are measured in both milliseconds and market moves. The recent From Rattle to Roar study tests this premise with MambaStock — a selective state-space model — trained to forecast S&P 500 weekly returns. The twist: pitting eight widely-used optimizers against a new family called Roaree, designed to capture Lion’s speed while taming its instability.
Why Optimizers Matter More in Finance Than You Think
In financial forecasting, milliseconds can mean the difference between execution and regret. This makes optimizer choice not just a theoretical concern but a practical lever for profitability. The study reinforces that adaptive-rate, momentum-based methods (Adam, RMSProp, Nesterov) deliver the lowest test errors for noisy, small-magnitude financial returns. Vanilla SGD struggles in this regime; AdamW’s decoupled weight decay over-regularizes, slowing convergence in already weak-signal environments.
Lion’s Roar — and its Flaws
The Lion optimizer, introduced for efficiency and robustness, shows a wide tolerance for hyperparameters and some of the fastest epoch times in the benchmark. This makes it appealing for large-scale experiments where speed matters. But on MambaStock, Lion exhibits oscillatory convergence — overshooting near the optimum and leaving performance on the table.
Enter Roaree: Smooth Moves in the Trading Jungle
Roaree modifies Lion’s non-differentiable sign step with smooth surrogates — tanh, arctan, softsign, sigmoid, erf, and norm — each controlled by a curvature parameter κ. The goal: dampen oscillations without sacrificing Lion’s speed. In head-to-head tests:
| Optimizer | Speed | Lowest Test MSE | Convergence Stability |
|---|---|---|---|
| Adam | Medium | Best Overall | Very Smooth |
| RMSProp | Medium | Near-Best | Smooth |
| Lion | Fastest | Mid-Tier | Oscillatory |
| Roaree (erf, κ=10) | Fastest+ | Better than Lion | Smooth |
The standout was Roaree with erf(κ=10) — trimming epoch time below Lion’s while improving test error. Most surrogates, except norm, noticeably stabilized training curves.
Practical Takeaways for Quant Teams
- Match Optimizer to Market Noise – In low-signal, high-noise data, gradient smoothing and adaptive learning rates matter more than raw speed.
- Speed Still Sells – Lion’s resilience to aggressive hyperparameters makes it a strong candidate for rapid hypothesis testing; Roaree extends this edge without the stability trade-off.
- Small Changes, Big Gains – Replacing a single non-differentiable function in the update rule yielded tangible improvements — a reminder that optimizer design is a high-leverage, under-explored domain in finance.
Beyond Roaree — The Next Frontier
The authors hint at integrating second-order methods like Sophia for curvature-aware updates, potentially doubling convergence speed. For real-world quant teams, this could mean faster model iteration cycles and more frequent strategy recalibration — especially critical when market regimes shift.
Bottom line: Optimizer choice isn’t a back-office technicality — it’s part of the alpha equation. Roaree’s smoother, faster convergence makes it an attractive addition to the quant’s toolbox, especially for models like MambaStock where sequence length and signal sparsity demand both efficiency and finesse.
Cognaptus: Automate the Present, Incubate the Future