Reading Between the Generated Lines: What AI Emotion Scores Really Measure
TL;DR for operators A conversational AI assistant may correctly identify a character’s emotions in a benchmark while remaining unreliable when a customer expresses frustration or concern indirectly. In ToMStories, the strongest open-weight model achieves 88.4% multiple-choice accuracy, above a small human baseline of 80.3%. Yet the highest model score on open-ended questions is 58.6%, below the human baseline of 64.2%. ...