Llm-Training

Workflow automation has a bad habit of looking impressive right up to the moment it touches reality. A demo agent can summarize a refund policy, draft a polite message, and call a refund_order() tool with great confidence. Then the real workflow asks a boring question: does this order exist, is it within the refund window, has it already been refunded, does the customer’s loyalty tier matter, and should the database state change after approval? ...

Freeze Now, Learn Faster: When Parameter Freezing Meets Pipeline Reality Freeze. That sounds like the least exciting verb in machine learning. We prefer more heroic verbs: scale, align, reason, distill, orchestrate, agentify. Freeze sounds like something a GPU does right before the invoice becomes spiritually educational. But in large-model training, freezing can be a serious efficiency tool. The idea is simple: if some parameters do not need to be updated at every step, skip their backward computation and save time. The trap is also simple: saving computation is not the same as saving wall-clock time. In pipeline-parallel training, a GPU can compute less and still finish the batch no earlier, because another dependency is blocking the schedule. Congratulations, the model learned less and the training job did not get meaningfully faster. A tiny miracle of systems inefficiency. ...

Payroll is a useful metaphor for agent training because it makes the absurdity obvious. Imagine a project team where one employee finds the right database, another enters the correct query, a third repeatedly calls the wrong API, and a fourth finally writes the report. If the report is accepted, everyone receives the same bonus. If it fails, everyone receives the same blame. Very democratic. Also very stupid. ...

Duplicates are supposed to be boring. In data engineering, duplicate records are usually treated as a hygiene problem: remove them, clean the pipeline, reduce noise, move on. In language-model training, repetition is less innocent. Repeated text can help a model learn an underrepresented domain. It can also teach the model to reproduce specific sequences too well. Somewhere between “useful exposure” and “verbatim recall,” a model stops learning only the pattern and starts carrying around the document. ...

An analyst opens a promising webpage. It contains the answer somewhere between a navigation menu, several years of archived material, an interactive table, related articles, legal disclaimers, and enough decorative HTML to keep a language model occupied until lunch. A human scans, clicks, ignores, and moves on. A browser agent is more likely to ingest the entire page, append it to an already swollen context window, and then congratulate itself for having “conducted research.” ...

Rollouts are expensive little creatures. They consume GPU time, produce long reasoning traces, wait for reward computation, and then—if the reward signal is flat—contribute exactly nothing to learning. The GPU was busy. The training dashboard looked serious. The model learned no usable distinction. Very productive, in the same way a meeting with twelve people and no decision is productive. ...

TL;DR for operators Training data is not a warehouse inventory problem. It is closer to nutrition. What helps a model early in pretraining may not be what helps it later, and a sample’s value can depend on the other samples sitting in the same batch. Obvious, perhaps. Operationalised? Less often. The paper behind this article, LLM Data Selection and Utilization via Dynamic Bi-level Optimization, proposes a Data Weighting Model, or DWM, that does not merely decide which data enters training. It assigns weights to samples within each batch, freezes those weights while the language model trains for a stage, then updates the weighting model using validation performance through a bi-level optimisation loop.1 ...