Online Learning

TL;DR for operators A UAV moving repeatedly between suburban and densely built areas encounters communication conditions it has experienced before. Yet a conventional online predictor may have forgotten those conditions while adapting to more recent measurements. LiLib retains small, environment-specific regression models and reactivates them when familiar conditions return. In simulations, this reduces prediction RMSE during the first 50 observations after a revisit from 12.29 to 5.70 dB relative to a sliding-window baseline. The improvement also carries into simulated throughput and outage, while four learned models can be transferred between UAVs using 480 bytes. The operational opportunity is selective model reuse rather than repeated retraining, provided real-world channels exhibit sufficiently recognizable recurring regimes. ...

TL;DR for operators An AI monitor that performs well before an agent begins learning may become unreliable as the agent adapts to its judgments. Monitor accuracy is therefore an ongoing property of the worker-monitor interaction, not merely a deployment-time benchmark. Research on continuously co-trained monitors establishes a theoretical condition under which occasional reliable corrections can support increasingly accurate oversight while the fraction of externally verified actions approaches zero. When legitimate work also remains sufficiently rewarding, better monitoring can discourage cheating rather than merely detect it. ...

TL;DR for operators Preference labels are usually budgeted as a quantity: buy more comparisons, improve the model further. Efficient Exploration at Scale suggests that this accounting misses a major variable—the value of the next comparison depends on which model produced it and whether the preference is actually uncertain.1 In the paper’s Gemma 9B pipeline, information-directed exploration reaches with fewer than 20,000 preference choices a win-rate level that offline RLHF requires more than 200,000 choices to reach. That is a directly observed efficiency improvement greater than 10x within the experiment. ...

TL;DR for operators A multi-worker LLM service can make a request cheaper by sending it to a worker that already holds much of the required prefix in KV cache. The problem is that repeatedly favoring that worker can create a queue. Spreading traffic more evenly reduces congestion but may throw away the computational advantage of prefix reuse. ...

Robots are easy to imagine as individuals. A quadruped walks. A drone flies. A warehouse arm picks. The business slide is usually kind enough to show one machine, one task, one satisfying arrow from input to output. Reality is less polite. A quadruped is not one decision-maker. It is a committee of limbs negotiating with gravity. A multi-drone system is not one policy with four propellers. It is a moving argument about timing, local perception, shared goals, and what not to crash into. A factory cell with multiple robotic agents is even worse: every local action changes the environment other agents are trying to understand. ...

Capacity looks simple until someone pays to jump the queue. That is the quiet problem behind a large amount of modern AI infrastructure. A platform may have many model instances, edge servers, or compute nodes. Tasks arrive with different business value. Enterprise traffic is more important than free-tier traffic. Some jobs have tighter latency targets. Some users, by contract or politics, are simply not equal. Lovely democratic fiction ends at the load balancer. ...

Ask an enterprise chatbot the wrong question on the wrong day and the problem is rarely that the language model has forgotten how to write English. The problem is that it has been handed the wrong pile of evidence. That is the expensive little defect inside many retrieval-augmented generation systems. The model may be fluent. The corpus may be current. The vector database may be humming along like a well-funded filing cabinet. Yet the answer still disappoints because the system chose the wrong snippets, placed a useful document too low, missed a newly relevant runbook, or treated yesterday’s user intent as if it were carved into basalt. ...

TL;DR for operators Production data does not politely wait for quarterly retraining. Sensor readings shift, fraud patterns mutate, market microstructure changes, network traffic acquires new habits, and customer behaviour performs its usual interpretive dance. This is concept drift: the model is still running, but the world it learned from has moved on. ...