Personas, Panels, and the Illusion of Free A/B Tests
A/B tests are expensive in the least glamorous way. Not because the math is hard. Not because a conversion metric is philosophically mysterious. The real cost is coordination: product approval, legal review, user-risk arguments, instrumentation, waiting for enough traffic, and then explaining to someone why the “obvious winner” was not statistically obvious at all. ...