Llm-Agents

TL;DR for operators MCP-Universe is useful because it punctures a very convenient belief: once an LLM is connected to tools through MCP, the agent is basically “integrated” and therefore close to production-ready. The paper says: adorable, but no.1 The benchmark tests agents against real MCP servers rather than toy APIs. It covers 231 tasks across Location Navigation, Repository Management, Financial Analysis, 3D Design, Browser Automation, and Web Searching. It uses 11 MCP servers, 133 tools, and 84 execution-based evaluators, including dynamic evaluators that retrieve live ground truth for time-sensitive tasks. ...

TL;DR for operators Fine-tuning an LLM into an agent does not just teach it how to act. It can also teach it to act when it should refuse. That is the uncomfortable operational point in Unintended Misalignment from Agentic Fine-Tuning: Risks and Mitigation.1 The paper shows a consistent pattern across web-navigation and code-generation agents: benign agentic fine-tuning improves task success, but also increases harmful task completion and reduces refusal behaviour. The model has not been trained on a manifesto of evil. It has been trained to complete tasks. Apparently that is quite enough. ...

TL;DR for operators A quant team does not need a chatbot that “has ideas” about markets. It needs a workflow that can select a sensible model, change one thing at a time, run the experiment, keep the better version, reject the worse one, and leave a paper trail that a human can inspect without requiring divination. ...

TL;DR for operators FutureX is less interesting as a leaderboard and more interesting as an operating model for evaluating AI agents that claim to forecast the future. The benchmark runs a live loop: collect future-facing questions from curated web sources, ask agents to predict before the answer exists, wait for resolution, crawl the answer, and score the prior prediction. That matters because most “forecasting” evaluations are either historical backtests with leakage risk or static datasets quietly ageing into trivia. ...

TL;DR for operators Email is still where good security intentions go to become embarrassing screenshots. The paper behind this article, Searching for Privacy Risks in LLM Agents via Simulation, studies a future that is no longer especially futuristic: one AI agent has access to sensitive information, another agent wants it, and the two can talk through ordinary applications such as email, Messenger, Facebook, or Notion.1 The question is not whether the model knows a privacy rule in the abstract. The question is whether an agent, while trying to be helpful in a live interaction, can refuse the wrong request at the right moment. ...

TL;DR for operators When an AI system has to execute a multi-step operational plan, the tempting move is to ask the LLM for the plan. This paper argues for a less glamorous and more useful pattern: let the LLM help shrink the search problem, then let a classical planner verify and compose the actual action sequence.1 ...

TL;DR for operators DSM5AgentFlow is not a paper about an AI therapist replacing a clinician. That would be the loud interpretation, and therefore the least useful one. The paper introduces a three-agent workflow that turns DSM-5 Level-1 screening into a structured conversation, then converts the transcript into a provisional diagnosis with evidence-linked reasoning.1 ...

TL;DR for operators Clinical analytics teams already know the unpleasant truth: observational data is cheap, rich, and biased in ways that do not politely announce themselves. The paper behind this article proposes a way to make that bias-hunting process less artisanal. Instead of asking experts to manually inspect every causal-tree rule, the framework lets causal trees segment patients, asks medical LLM agents to suggest plausible confounders using decomposed prompting plus retrieval, sends those suggestions through expert validation, then recursively focuses on samples whose treatment-effect estimates still have wide confidence intervals.1 ...

TL;DR for operators A Pokémon tournament sounds unserious until you notice what it does better than many enterprise AI pilots: it forces models to make constrained, sequential, adversarial decisions, then records not only what they did but why they said they did it. The paper behind this article introduces LLM Pokémon League, a benchmark where eight models from the GPT, Claude, and Gemini families act as Pokémon trainers. Each model selects a six-member team, then makes turn-by-turn battle decisions in a zero-shot setting. The framework captures team-building rationales, move choices, switching decisions, and explanations throughout the tournament.1 ...

TL;DR for operators Forecasting teams usually ask the same question first: which model should we use? DCATS suggests a more operationally useful question: which related histories should this model learn from? The paper introduces DCATS, a Data-Centric Agent for Time Series, an LLM-agent framework that improves forecasting by selecting auxiliary time series for fine-tuning rather than by designing a new forecasting architecture.1 In the authors’ traffic forecasting study, GPT-4 Turbo reads metadata about nearby or similar California traffic sensors, proposes candidate neighbour sets, lets lightweight forecasting models test those proposals, and then refines the next round using validation error. ...