Attention Sinks

Serving an LLM is usually discussed in pleasantly managerial language: latency, throughput, context windows, GPU memory, quantization, cache eviction. Nice clean nouns. Then the model ruins the spreadsheet by producing internal activations that are thousands of times larger than ordinary values, while some tokens quietly become attention magnets for reasons that are not exactly semantic. Very professional behavior from a trillion-dollar technology stack. ...

Fine-tuning is supposed to be the polite part of AI customization. A company uploads domain data. A provider adapts an aligned model. The final model still refuses harmful requests, still answers useful questions, and ideally becomes more competent at the client’s narrow task. Everyone nods. The demo works. The governance slide says “safety preserved.” The slide, as usual, is doing a lot of unpaid labor. ...