Optimization

Opening — Why this matters now Modern deep learning quietly assumes a comforting fiction: that training is memoryless. Given the current parameters (and maybe the optimizer buffers), tomorrow’s update shouldn’t care about yesterday’s data order, augmentation choice, or micro-step path. This assumption underwrites theory, stabilizes intuition, and keeps whiteboards clean. Reality, however, has been less cooperative. Practitioners know that order matters, momentum carries ghosts of past gradients, and small curriculum tweaks can echo far longer than expected. Yet until now, there has been no clean, operational way to measure whether training truly forgets—or merely pretends to. ...

Opening — Why this matters now Multi-agent LLM systems are everywhere: debate frameworks, critic–writer loops, role-based agents, orchestration layers stacked like an over-engineered sandwich. Empirically, they work. They reason better, hallucinate less, and converge on cleaner answers. Yet explanations usually stop at hand-waving: diversity, multiple perspectives, ensemble effects. Satisfying, perhaps—but incomplete. This paper asks a sharper question: why do multi-agent systems reach solutions that a single agent—given identical information and capacity—often cannot? And it answers it with something rare in LLM discourse: a clean operator-theoretic explanation. ...

Opening — Why this matters now Everyone wants LLMs to think harder. Enterprises, however, mostly need them to think correctly — especially when optimization models decide real money, real capacity, and real risk. As organizations scale, optimization problems grow beyond toy examples. Data spills into separate tables, constraints multiply, and naïve prompt‑to‑solver pipelines quietly collapse. ...

Opening — Why this matters now LLM inference has quietly become the dominant cost center of modern AI systems. Training grabs headlines; inference drains budgets. As models scale into the tens of billions of parameters, every additional forward pass hurts — financially and operationally. Speculative decoding promised relief by letting small models run ahead and big models merely verify. But verification, ironically, became the bottleneck. ...

Opening — Why this matters now Traffic simulation has always promised more than it delivers. City planners, transport researchers, and policymakers are told that with the right simulator, congestion can be eased, emissions reduced, and infrastructure decisions made rationally. In practice, most simulators demand deep domain expertise, rigid workflows, and a tolerance for configuration pain that few real-world users possess. ...

Opening — Why this matters now Modern deep learning training is an odd contradiction. We obsess over architectures, data curation, and trillion-token scaling laws—then quietly accept Cosine Annealing as if it were gravity. Learning rate schedules are often inherited, not argued for. This paper challenges that complacency with a scheduler that does something almost offensive in its simplicity: it just watches the loss and reacts. ...

When agent frameworks stall in the real world, the culprit is rarely just a bad prompt. It’s the wiring: missing validators, brittle control flow, no explicit state, and second-hop retrieval that never gets the right handle. Maestro proposes something refreshingly uncompromising: optimize both the agent’s graph and its configuration together, with hard budgets on rollouts, latency, and cost—and let textual feedback from traces steer edits as much as numeric scores. ...

TL;DR Pair a classic mixed‑integer inventory redistribution model with an LLM-driven context layer and you get explainable optimization: the math still finds near‑optimal transfers, while the LLM translates them into role‑aware narratives, KPIs, and visuals. The result is faster buy‑in, fewer “why this plan?” debates, and tighter execution. Why this paper matters for operators Most planners don’t read constraint matrices. They read stockout risks, truck rolls, and WOS. The study demonstrates a working system where: ...

TL;DR Classical planners crack under scale. You can rescue them with LLMs in two ways: (1) Inspire the next action, or (2) Predict an intermediate state and split the search. On diverse benchmarks (Blocks, Logistics, Depot, Mystery), the Predict route generally solves more cases with fewer LLM calls, except when domain semantics are opaque. For enterprise automation, this points to a practical recipe: decompose → predict key waypoints → verify with a trusted solver—and only fall back to “inspire” when your domain model is thin. ...

If federated fine-tuning feels like trying to teach calculus to a toddler on a flip phone, you’re not alone. While the privacy-preserving benefits of federated learning are clear, its Achilles’ heel has always been the immense cost of training large models like LLaMA2-13B across resource-starved edge devices. Now, a new method—DEVFT (Developmental Federated Tuning)—offers a compelling paradigm shift, not by upgrading the devices, but by downgrading the expectations. At least, at first. ...