Explain Before You Explore: What YouTube Music’s LLM Experiment Changed
TL;DR for operators YouTube Music tested an LLM-enhanced discovery experience and reported a 22.43% lift in shelf-level engagement and an 8.07% lift in its discovery-retention metric. The tempting explanation is that the LLM simply found better unfamiliar artists. The component tests point elsewhere: withholding the new nomination source alone had a neutral engagement impact, while removing personalized natural-language rationales produced regressions resembling removal of the complete LLM system. ...