One Harness, Many Failure Modes
TL;DR for operators A platform team supporting several language models can achieve respectable task coverage while still delivering inconsistent results across those models. The relevant question is not only whether some supported model completes a task, but whether the same agent harness works reliably with every supported model. MaD-RSI, proposed by Yu Cui and colleagues,1 addresses this problem by using execution successes and failures from heterogeneous, frozen LLMs to improve one shared harness. On MedXpertQA, it matches the strongest single-model optimization baseline’s 60% four-model success coverage while raising joint success from 13% to 18%. ...