Enterprise AI

TL;DR for operators An enterprise assistant may need to reach an answer through an internal identifier or intermediary record that shares almost no wording with the user’s question. Semantic similarity can recognize a plausible final answer while suppressing the unremarkable bridge entity needed to reach it. Larger retrieved neighborhoods do not solve this automatically; they can expose more valid paths while adding more convincing distractions. ...

TL;DR for operators A visual support assistant shown an unfamiliar machine, product, bird, or venue cannot answer by retrieval alone. It must first determine what the image depicts, then locate the missing fact, while deciding whether another search is worth the delay. A wrong first match can redirect every later step toward the wrong entity. ...

TL;DR for operators An AI failure is an observation, not a diagnosis. A low benchmark score, an incorrect answer, or a broken agent run does not tell you whether the underlying problem belongs in the training data, model objective, retrieval policy, procedural instructions, tool interface, or execution environment. Treating all of these as “model quality” produces expensive interventions with weak causal logic. ...

TL;DR for operators When an AI system reads a manual, policy, API specification, case file, or operating procedure, finding the relevant facts is only half the job. It must also discover the local rules that define what a valid answer looks like: required fields, exact labels, ordering constraints, exception handling, validation steps, prohibited actions, and completeness conditions. ...

TL;DR for operators The usual response to an unreliable AI system is to ask for a larger model. That is frequently an expensive way to avoid diagnosing the actual error. Three recent papers point to a more disciplined alternative: Rules that must never be violated should be enforced at generation time, not merely suggested in a prompt. Stable patterns in the problem domain should be built into the model architecture, so the model does not have to rediscover them from every dataset. Residual temporal, class, and modality errors may be better handled through calibration, smoothing, routing, and fusion than through another round of full-model training. These interventions provide different kinds of assurance. A grammar mask can make certain outputs unreachable. An architectural prior can make desirable patterns more likely. Calibration can improve observed performance but usually cannot guarantee behavior. Bigger models still matter where genuine semantic reasoning is required. The lesson is not “small beats large.” It is “do not pay a large model to solve a problem that a rule, prior, or threshold can solve more reliably.” For business leaders, this changes the architecture question from “Which model should we buy?” to “Which layer should own each requirement?” ...

TL;DR for operators Production AI does not become reliable by remembering everything. It becomes reliable by preserving the information that defines the current state, explicitly representing what is allowed to change, and discarding history that would contaminate the next decision. Two papers arrive at this conclusion from remarkably different directions. One generates future amyloid-PET scans by anchoring the generative process to a patient’s baseline scan. The other builds persistent enterprise agents by retaining specifications, schemas, tools, and output contracts while deleting prior reasoning traces. ...

TL;DR for operators A retrieval system does not become trustworthy merely because it has documents. It becomes a system with several possible ways to be confidently wrong. MACR treats disagreement as an adjudication problem. It estimates whether the model appears to know the answer, turns that internal position into inspectable text—or retrieves an external substitute when confidence is low—then asks specialized agents to identify contradictions and apply validated resolution rules. ...

TL;DR for operators Red teaming is not the act of finding one clever prompt that makes a model misbehave. That is a demo. Sometimes a useful demo, occasionally a terrifying one, but still a demo. The two papers here point to something more operational. RECAP shows how adversarial prompt generation can become cheaper by retrieving previously successful attack patterns rather than optimizing every new attack from scratch.1 A separate red-teaming framework shows how those attacks can be routed through a controlled attacker-target-jury workflow, with ensemble judging, task-specific criteria, and cross-linguistic analysis.2 ...

TL;DR for operators The current enterprise LLM mistake is to treat safety as a chat-interface problem. Put a filter in front, add a stern system prompt, run a cheerful demo, and hope the model has suddenly acquired a moral philosophy and a compliance department. Charming. Also insufficient. Two recent papers make a stronger and more operationally useful point. The first evaluates open-source LLMs under prompt-injection and jailbreak attacks, then compares lightweight inference-time defenses such as input filtering, system-prompt hardening, vector detection, voting, and self-examination.1 The second studies whether fine-tuned language models memorize sensitive personally identifiable information that appears only in inputs, not in the desired outputs, and benchmarks mitigation methods such as differential privacy, UnDial, regularization, and DPO.2 ...

TL;DR for operators Runbooks decay. APIs shift, data schemas mutate, file paths move, and the “expert procedure” that worked last quarter starts quietly steering an agent into a wall. The paper behind this article, SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing, asks a useful operational question: can an agent skill be improved when nobody has provided hidden tests, reference answers, task rewards, or expert labels?1 ...