Training Dynamics

Opening — Why this matters now For years, the dominant anxiety around large language models has been hallucination: the model makes things up. The paper you just read argues that we’ve been staring at the wrong failure mode. The real issue is subtler and arguably more dangerous: memorization sinks — regions of the training distribution where models stop learning general structure and instead collapse into rote recall. These sinks don’t merely inflate benchmark scores; they quietly reshape model behavior, evaluation outcomes, and downstream reliability. ...

Opening — Why this matters now Modern deep learning quietly assumes a comforting fiction: that training is memoryless. Given the current parameters (and maybe the optimizer buffers), tomorrow’s update shouldn’t care about yesterday’s data order, augmentation choice, or micro-step path. This assumption underwrites theory, stabilizes intuition, and keeps whiteboards clean. Reality, however, has been less cooperative. Practitioners know that order matters, momentum carries ghosts of past gradients, and small curriculum tweaks can echo far longer than expected. Yet until now, there has been no clean, operational way to measure whether training truly forgets—or merely pretends to. ...

Opening — Why this matters now For years, we have comforted ourselves with a tidy distinction: models generalize, databases memorize. Recent research quietly dismantles that boundary. As LLMs scale, memorization is no longer an edge case—it becomes a structural property. That matters if you care about data leakage, IP exposure, or regulatory surprises arriving late but billing retroactively. ...