Cover image

The Model Stayed Frozen. The Agent Got Better.

TL;DR for operators Designer-RSI shifts one production decision: improving an agent does not always require improving its foundation model. The paper keeps the model, tools, renderer, and evaluator frozen and instead changes an external library of reusable operating procedures. The strongest evidence is not simply that the library grows. On a fixed 200-brief ablation, adding new procedures alone produced a 49.4% win rate against the base agent, while rewriting existing procedures reached 48.6%. Combining both reached 58.5%, with completeness rising to 74.04. The operational lesson is that continual adaptation looks less like accumulating prompts and more like maintaining a system with separate processes for coverage, repair, retrieval, testing, and release. ...

October 14, 2026 · 7 min · Zelina
Cover image

When Helpful Starts Deciding for You

TL;DR for operators For subjective personal questions, the main risk is not limited to whether an AI gives a wrong answer. It is also whether the system helps a user form a judgment or supplies the judgment for them. Cheng and colleagues call the latter pattern LLM-as-oracle use.1 Their evidence spans public conversation logs, donated longitudinal histories, a preregistered experiment, a randomized three-week interaction study, and response-policy interventions. ...

October 14, 2026 · 7 min · Zelina
Cover image

Human-Like Without Engagement: The Measurement Trap in LLM Psychometrics

TL;DR for operators Suppose an AI team administers a standardized values questionnaire to several models, compares their scores with a target population, and selects the model with the smallest statistical distance. That procedure has a failure mode: a model can look close to the human average precisely because it is producing flat or middle-of-the-scale answers rather than responding coherently to the questions. ...

October 3, 2026 · 7 min · Zelina
Cover image

A Proof Can Pass and Still Mean the Wrong Thing: What AxQM Changes About Formal AI Evaluation

TL;DR for operators When an AI system produces a formal proof, an evaluation team can make one part of the verdict unusually objective: either the proof is accepted under the allowed rules, or it is not. That removes much of the grader variance found in rubric scoring or LLM judging. AxQM provides 1,019 Lean 4 proof-synthesis tasks over 479 finite-dimensional quantum-mechanics textbook items. It keeps a private reference solution for every task and grades submissions through successful compilation, absence of sorry in the proof or its dependencies, and absence of newly introduced axioms.1 ...

September 22, 2026 · 6 min · Zelina
Cover image

Think Harder, See Less? What Visual Illusions Reveal About Multimodal Reasoning

TL;DR for operators Giving a vision-language model more time to reason sounds like a reliability upgrade. On deceptive visual tasks, that assumption is too broad. Across seven model configurations tested with and without additional deliberation, longer reasoning improved free-form explanations of why an illusion occurs, yet detection accuracy fell for several Qwen models and multiple-choice reasoning fell for several open-source systems. ...

September 14, 2026 · 7 min · Zelina
Cover image

When Model Output Can Change State: An Architecture Guide to Agent Reliability

TL;DR for operators A tool-using agent does more than generate an answer. It observes part of a workflow, carries information forward, decides what to do, changes external state, and then reacts to the result. A wrong answer in a chatbot may remain text; a wrong action in an agent can alter a file, submit a transaction, call the wrong service, or create a bad state that later decisions treat as valid. ...

September 6, 2026 · 7 min · Zelina
Cover image

Strong Draft, Incomplete Handoff: What BusinessCaseBench Says About Business Analysis

TL;DR for operators A manager can receive an AI-generated business analysis that looks strong and still have a good reason to require another reviewer. In BusinessCaseBench, Claude Sonnet 4.6 and GPT-5.4 receive credit for covering most expected analytical elements—88.4% and 87.2% respectively—but fully satisfy every required element on only 49.6% and 47.6% of questions. ...

August 13, 2026 · 8 min · Zelina
Cover image

Five Answers, One Bad Retrieval: When RAG Agreement Misleads

TL;DR for operators A production RAG system returns the same answer five times. The operator still has to decide whether to release it, investigate it, or send it for review. Repeated agreement is useful evidence that generation is stable, but it does not show that the system retrieved the right material. All five answers may have been generated from the same empty, incomplete, or incorrect context. In that case, repeated sampling does not independently test the answer; it repeatedly tests the decoder under one defective retrieval state. A wrong answer that remains effectively unchanged across samples is a silent error. ...

July 30, 2026 · 10 min · Zelina
Cover image

Measure for Measure: Why AI Evaluation Must Follow the Failure

TL;DR for operators A lower model bit width is not automatically a speedup. A lower training loss is not automatically a reliable policy. GRINQH evaluates quantization through the mechanism it is meant to change: decoding-stage memory traffic, kernel throughput, end-to-end generation speed, and retained task accuracy.1 Kolmogorov regression evaluates diffusion policies through trajectory geometry, a PDE-based inference residual, rollout behavior, anomaly detection, and an external safety filter.2 The shared lesson is not that the two forms of “precision” are technically equivalent. They are not. The lesson is that fidelity and evidence should be allocated according to the actual failure structure of the system. A production evaluation should connect four things explicitly: the intervention, the mechanism it changes, the diagnostic that observes that change, and the operational outcome that justifies deployment. Composite scores are useful only when their weights reflect real business priorities and their components remain separately visible. Otherwise, they are merely spreadsheets wearing authority. The dashboard is not the system AI evaluation has developed an awkward habit: optimize a convenient number, improve that number, and declare the system improved. ...

July 22, 2026 · 16 min · Zelina
Cover image

Route Cause Analysis: Stop Sending Every AI Failure to Training

TL;DR for operators An AI failure is an observation, not a diagnosis. A low benchmark score, an incorrect answer, or a broken agent run does not tell you whether the underlying problem belongs in the training data, model objective, retrieval policy, procedural instructions, tool interface, or execution environment. Treating all of these as “model quality” produces expensive interventions with weak causal logic. ...

July 22, 2026 · 19 min · Zelina