Cover image

A Different Forecast Without a Different Model

TL;DR for operators A utility forecasting hundreds of heterogeneous customers and distribution transformers faces a model-management decision: maintain one shared forecaster, train separate local models, or specialize a common model without multiplying its full parameter set. Li, Cheng, and Weng demonstrate a third architecture. Their method keeps a probabilistic forecasting backbone fixed and constructs customer-specific predictive mean and variance heads from a shared bank of low-rank adaptations. Customers can combine several adapters according to their load characteristics rather than receiving an independently trained model. ...

November 11, 2026 · 8 min · Zelina
Cover image

When One Merged Model Is One Too Few

TL;DR for operators Combining eight specialized AI adapters into one deployable adapter can reduce infrastructure complexity, but it can also discard capabilities when the adapters encode poorly aligned parameter updates. Di Wu and Shuaidong Pan’s Group-wise Rank-Aware Model Merging (GRAM)1 addresses this by clustering compatible task updates before merging them. It also allocates limited representational capacity according to the strength of each task’s learned directions. The merging procedure requires model weights, not task training or calibration data. ...

October 29, 2026 · 9 min · Zelina
Cover image

The Smallest Client Needs More Than a Better Backbone

TL;DR for operators A pretrained model can be well suited to a text-classification task and still perform poorly for a participant with almost no labeled data. In experiments reported by Naseer and colleagues,1 removing federated aggregation reversed the pretrained models’ worst-client advantage across all five tested configurations. Personalization recovered only 4–12% of the performance gap between local-only and shared-model training in three domain-matched settings. ...

October 27, 2026 · 9 min · Zelina
Cover image

Training the Next Skill Can Weaken the Last

TL;DR for operators Adding a new capability to an LLM agent can substantially degrade capabilities it previously acquired. In ACLArena, training an agent for e-commerce transactions improved retail performance while sharply reducing its mathematical reasoning and search scores. The researchers compare several ways to preserve capabilities and introduce Mixture of Low-Rank Experts (MLE): a shared supervised foundation with separately trained, low-rank reinforcement-learning adapters. MLE achieves a more balanced performance profile than the evaluated consolidation baselines across several benchmarks. ...

October 26, 2026 · 8 min · Zelina
Cover image

Spend the Second Write Where It Pays

TL;DR for operators A continuously updated LLM does not necessarily need to modify its parameters for every new fact. If retrieval already makes an update answerable, the expensive second write may add little. ALLOT, a hybrid-memory routing method from Huang, Fang, Xiao, and Du,1 keeps every factual update in external retrieval memory and ranks updates for additional parametric adaptation. A hard cutoff then assigns the available adaptation budget to only the highest-ranked facts. ...

October 19, 2026 · 7 min · Zelina
Cover image

Teaching the Query When to Move: Small Models as Retrieval Controllers

TL;DR for operators A retrieval agent does not merely retrieve documents. It repeatedly decides what to do next: search again, rewrite a query, record evidence, combine what it has found, verify a claim, or stop. The Fellowship of the Query: Learning Retrieval Actions1 shows that these decisions can be taught to small language models as a supervised classification problem. Across the evaluated models, LoRA fine-tuning sharply improves prediction of the next retrieval action. For Granite 4.1 3B, macro-F1 rises from 0.1736 zero-shot to 0.6536 after fine-tuning. ...

October 3, 2026 · 7 min · Zelina
Cover image

Probe Before You Prune: Measuring Which MoE Experts a Workload Can Lose

TL;DR for operators A mixture-of-experts model can be too large for an inference budget even though only a small subset of its experts executes for each token. The inactive experts still have to reside somewhere. In the measured Mixtral deployment studied by Janati et al., removing four of eight experts per layer cut 4-bit memory from 24.2 GB to 12.3 GB and per-token latency from 40.3 ms to 25.2 ms on a single A100.1 ...

September 6, 2026 · 9 min · Zelina
Cover image

The Optimizer Comes Before the Quantizer

TL;DR for operators A small model can perform acceptably after task specialization and still lose more accuracy than expected when compressed. The paper examined here suggests that part of that loss may be shaped earlier, during fine-tuning, rather than being determined by the quantizer alone. Paper evidence: Muon-optimized models lost less accuracy after quantization than Adam-optimized models on six of eight benchmarks. On ARC-e, for example, accuracy fell by 0.55 percentage points after quantization with Muon versus 3.16 points with Adam. The integrated pipeline also produced a 2.86 GB quantized model from a 6.01 GB pre-quantized model while improving reported serving throughput and roughly halving per-token latency. ...

September 6, 2026 · 8 min · Zelina
Cover image

When Confidence Drives the Workflow: What HypeLoRA Changes About Adapter Selection

TL;DR for operators A classifier can score better on a benchmark while giving probabilities that systematically overstate or understate how often it is correct. That matters whenever confidence feeds an escalation threshold, automatic action, human-review queue, or other operational rule. LoRA adapts a frozen model through a small update built from two low-rank factors, $A$ and $B$. HypeLoRA asks whether generating those adaptations across layers with a shared hyper-network improves not just task performance but the reliability of the resulting confidence signal.1 The answer is conditional: standard LoRA does not calibrate uniformly better than full fine-tuning, and fully generated HypeLoRA remains broadly similar to ordinary LoRA. The strongest reported calibration result comes from the Transformer fixed-$A$ configuration, which freezes $A$ and generates only $B$: ECE reaches 0.100 on CoLA and 0.028 on SST-2, while task performance remains below the strongest-performing configuration. ...

September 6, 2026 · 7 min · Zelina
Cover image

Compress the Activation, Spend the Memory on Rank

TL;DR for operators A LoRA fine-tuning job can fit its trainable parameters comfortably on a GPU and still run out of memory because backpropagation retains large intermediate activations. CARE-LoRA attacks that remaining buffer rather than shrinking the adapter itself. Zhang et al. store LoRA’s already-compressed activation plus a small reconstruction matrix, then use those tensors to approximate only the gradient needed for one of LoRA’s two trainable matrices.1 ...

September 5, 2026 · 7 min · Zelina