Vision Transformers

TL;DR for operators Many production AI systems do not need a more poetic answer. They need a cheaper way to decide whether the answer should be trusted at all. The paper introduces Calibrated Variance Propagation (CVP), a test-time method for Bayesian deep learning that estimates predictive uncertainty without repeatedly sampling model weights through many forward passes.1 It targets a practical bottleneck: recent variational training methods can now produce Gaussian weight posteriors for large neural networks at training costs comparable to standard optimizers, but using those posteriors at inference usually means Monte Carlo sampling. That is expensive, especially when the model must respond in real time. Apparently, reliability is still expected to fit inside latency budgets. Outrageous. ...

Updating a model sounds harmless until the old parts of the system start reading the new representations incorrectly. That is the less theatrical version of catastrophic forgetting. Not the dramatic story where a neural network “forgets everything” like a distracted intern. The more useful story is quieter: a deployed AI system adapts its backbone to new data, the feature space shifts, and classifiers trained for earlier tasks are left calibrated to yesterday’s geometry. ...

Images are expensive. Not emotionally, although some product managers do try. They are expensive because modern visual models turn an image into a sequence of tokens, then let those tokens attend to one another. In a Vision Transformer, more tokens usually mean more detail, but also more attention cost. The obvious response is to reduce the number of tokens. ...