Motivation Is Something Your Models Need: When Curiosity Becomes a Training Strategy
Training budgets are where elegant architecture slogans go to be audited. The usual response to a model that needs better accuracy is painfully familiar: make it larger, train it longer, feed it more data, and then pretend the GPU bill is a philosophical problem. The paper Motivation Is Something You Need takes a more interesting route. It asks whether a model needs to be large all the time, or whether extra capacity can be activated only when training signals suggest the model is “getting somewhere.”1 ...