LLM Personalization

A customer tells your AI assistant that she dislikes crowded tourist attractions. Three weeks later, she asks for a weekend itinerary. A good assistant should not proudly recommend the busiest landmark in the city. A less good assistant will do exactly that, but in a warm tone. This is the quiet failure mode behind many “personal AI” demos. The interface remembers the conversation. The product claims continuity. The model may even have a giant context window large enough to swallow a small novel. Yet when the user asks a new question, the system behaves as if the earlier preference is just decorative text floating somewhere in the attic. ...

Personas have always been the slightly embarrassing cardboard cut-outs of product strategy. A marketing team invents “Sarah, 34, urban professional, values convenience.” A UX team adds “busy mother of two.” Someone in sales insists she is “budget-conscious but aspirational,” because apparently every fictional human being is. Then everyone nods solemnly and uses Sarah to justify a pricing page, an onboarding flow, or an ad campaign. ...

Memory is a useful product feature until it becomes a junk drawer. That is the quiet problem behind many “personalized” AI systems. A user has a history. The system retrieves some of it. The model receives a longer prompt. The output becomes, in theory, more personal. In practice, the assistant often behaves like someone who read your old emails in a hurry and decided this was the same as knowing you. ...

TL;DR for operators Personalization does not fail because the model forgot your birthday. That would be almost charming. It fails because the system remembers too much in the wrong shape. The Cupid benchmark tests whether LLMs can infer a user’s context-dependent preference from prior multi-turn interactions and apply it to a new request.1 The setup is deliberately business-relevant: users do not announce a clean preference profile; they reveal expectations through feedback, correction, and mild conversational friction. Very realistic. Nobody fills out a YAML file called my_deeply_contextual_preferences.yml, at least not outside certain Slack channels. ...