Online-Learning

TL;DR for operators Preference labels are usually budgeted as a quantity: buy more comparisons, improve the model further. Efficient Exploration at Scale suggests that this accounting misses a major variable—the value of the next comparison depends on which model produced it and whether the preference is actually uncertain.1 In the paper’s Gemma 9B pipeline, information-directed exploration reaches with fewer than 20,000 preference choices a win-rate level that offline RLHF requires more than 200,000 choices to reach. That is a directly observed efficiency improvement greater than 10x within the experiment. ...

TL;DR for operators A multi-worker LLM service can make a request cheaper by sending it to a worker that already holds much of the required prefix in KV cache. The problem is that repeatedly favoring that worker can create a queue. Spreading traffic more evenly reduces congestion but may throw away the computational advantage of prefix reuse. ...

Robots are easy to imagine as individuals. A quadruped walks. A drone flies. A warehouse arm picks. The business slide is usually kind enough to show one machine, one task, one satisfying arrow from input to output. Reality is less polite. A quadruped is not one decision-maker. It is a committee of limbs negotiating with gravity. A multi-drone system is not one policy with four propellers. It is a moving argument about timing, local perception, shared goals, and what not to crash into. A factory cell with multiple robotic agents is even worse: every local action changes the environment other agents are trying to understand. ...

Capacity looks simple until someone pays to jump the queue. That is the quiet problem behind a large amount of modern AI infrastructure. A platform may have many model instances, edge servers, or compute nodes. Tasks arrive with different business value. Enterprise traffic is more important than free-tier traffic. Some jobs have tighter latency targets. Some users, by contract or politics, are simply not equal. Lovely democratic fiction ends at the load balancer. ...

Ask an enterprise chatbot the wrong question on the wrong day and the problem is rarely that the language model has forgotten how to write English. The problem is that it has been handed the wrong pile of evidence. That is the expensive little defect inside many retrieval-augmented generation systems. The model may be fluent. The corpus may be current. The vector database may be humming along like a well-funded filing cabinet. Yet the answer still disappoints because the system chose the wrong snippets, placed a useful document too low, missed a newly relevant runbook, or treated yesterday’s user intent as if it were carved into basalt. ...

TL;DR for operators Production data does not politely wait for quarterly retraining. Sensor readings shift, fraud patterns mutate, market microstructure changes, network traffic acquires new habits, and customer behaviour performs its usual interpretive dance. This is concept drift: the model is still running, but the world it learned from has moved on. ...