Cover image

Farther Apart, Same Context

TL;DR for operators A transformer can perform well when the information it needs appears at familiar positions and still fail when the same dependency is moved farther away inside an unchanged context window. Nevermann and Gros isolate this problem with a fixed-context test of distance generalization.1 In their main 8-layer, 8-head, hidden-size-512 configuration, a transformer with no explicit positional encoding, or NoPE, generalizes better to unseen dependency distances than RoPE or ALiBi. But the result does not justify removing positional encoding by default: smaller NoPE models become fragile, while explicit positional schemes remain more stable. ...

October 16, 2026 · 7 min · Zelina
Cover image

The One-Step Mirage: Stress-Testing World Models Beyond the Next Frame

TL;DR for operators A world model can be reasonably accurate when predicting the next state and still become unreliable once its own predictions are fed back repeatedly. That difference is operationally relevant because planning and control systems often depend on trajectories, not isolated one-step estimates. Semigroup-JEPA: Latent Dynamics Consistency for Zero-Shot Physics Generalization1 tests this problem by training models on a narrow range of gravity values and evaluating them across much wider regimes. SG-JEPA jointly trains its visual representation and temporal predictor using several recursively generated future latents. Its largest advantages generally appear where errors have more opportunity to compound: longer forecast horizons and gravity values outside the training distribution. ...

October 3, 2026 · 7 min · Zelina
Cover image

HAROOD: When Benchmarks Grow Up and Models Stop Cheating

A wearable model can look brilliant in the lab and embarrass itself on Monday morning. The user changes. The watch slides down the wrist. A sensor is mounted on the chest instead of the pocket. The same person walks differently after fatigue, injury, aging, or simply because life has the terrible habit of not matching the training set. Human Activity Recognition, or HAR, has always lived with this problem. It turns sensor streams from accelerometers, gyroscopes, EMG, ECG, and other wearable or ambient devices into labels such as walking, running, sitting, cycling, or stress state. It is useful precisely because it moves into the real world. That is also where benchmark accuracy goes to die. ...

December 12, 2025 · 20 min · Zelina
Cover image

Noisy but Wise: How Simple Noise Injection Beats Shortcut Learning in Medical AI

X-rays look clinical. To a neural network, they can also look like stationery. A hospital name in the corner. A scanner signature. A compression pattern. A familiar positioning marker. A slightly different way of cropping the lung field. None of these is pneumonia. None of these is COVID-19. Yet a deep learning model trained on small medical datasets can treat them as wonderfully convenient diagnostic evidence, because machines are very good at passing exams and less naturally committed to understanding what the exam is about. ...

November 9, 2025 · 15 min · Zelina