Inference Cost

TL;DR for operators If your agent stack is becoming expensive because every “reflection” step means another model call, this paper is worth reading. Its proposal, Introspection of Thought (INoT), tries to compress an external multi-agent debate loop into one structured prompt. The LLM is not literally running multiple agents. It is being instructed, through a hybrid Python-and-natural-language prompt called PromptCode, to simulate two internal debaters that reason, critique, rebut, revise, and then return an answer.1 ...

TL;DR for operators Ada-R1 is useful because it attacks the expensive part of reasoning models from the right angle: not “make every answer shorter,” but “decide which problems deserve long reasoning in the first place.”1 The paper’s key evidence is uncomfortable for anyone buying premium reasoning capacity by default. Long Chain-of-Thought helps on harder mathematical problems, but nearly half of the analysed samples show no improvement from Long-CoT, and some perform worse. In other words, paying for the model to brood majestically over simple work is not intelligence. It is ceremony with a token meter attached. ...

TL;DR for operators The real AI cost question is not “Which model is cheapest?” It is “Which workflow delivers acceptable outcomes at the lowest verified total cost?” Token price is only the most visible line item. The less photogenic costs are retries, review, integration, monitoring, compliance, vendor lock-in, and the small corporate tragedy known as “we saved money on inference and spent it all on fixing nonsense.” ...