Benchmarking the Benchmarks: When AI Can’t Agree on the Rules
Benchmarks are supposed to settle arguments. In practice, they often create better-looking arguments. A logistics optimizer claims it balances distance, delivery time, fuel cost, and risk. A robot planner claims it can trade off speed against safety. A routing engine claims it returns not one answer, but a frontier of reasonable alternatives. Fine. Then comes the awkward question: tested on what? ...