More Data, Better Learner: What Children Reveal About Learning Efficiency

TL;DR for operators

Adding more data and becoming better at learning from data are different objectives.

Across five longitudinal vocabulary datasets covering American English, Norwegian, and Japanese, young children show strongly increasing returns to developmental experience. Using a matched per-word estimator, the language models examined in the study produce a median acceleration estimate of 1.16, with an interquartile range of 0.93–1.46. Corrected child estimates fall around 10.4–13.8.

The operational lesson is not that language models cannot learn words. They clearly do. The difference is in the shape of learning: the examined models continue to receive approximately constant proportional returns from additional data, while children become progressively more efficient.

For model-development teams working under fixed data or compute budgets, this suggests another metric alongside final loss: whether the marginal value extracted from later training examples is rising. That makes learning dynamics themselves an engineering target. The paper does not establish which mechanism would produce child-like acceleration in AI systems, nor does it show that every model architecture lacks it.

More training data can improve a model without improving the learner

When a training run receives another billion tokens, performance usually improves. But two systems can reach better performance through very different dynamics.

One system may extract approximately the same proportional benefit from each multiplicative increase in data. Another may become better at using new experience, so later input produces larger gains than earlier input.

Michael C. Frank’s Children, but not language models, show accelerating returns in word learning¹ asks whether this distinction helps explain part of the familiar data-efficiency gap between children and neural language models.

The strongest comparison comes from applying the same per-word learning-curve estimator to both. Across 609 evaluated words, the language-model median acceleration estimate is 1.16 [IQR 0.93, 1.46]. After correcting for attenuation in the comparable child estimator, the five child datasets yield values from roughly 10.4 to 13.8.

Only after establishing that contrast does the paper introduce its key parameter, $\kappa$. A value of $\kappa=1$ corresponds to pure accumulation: more experience helps, but the learner does not become progressively more efficient. Values above 1 indicate acceleration.

The child estimates are not marginally above that benchmark. They are an order of magnitude larger.

The model separates knowing easier words from learning faster

Rapid vocabulary growth alone would not establish increasing learning efficiency. A child could simply know more words because more time has passed, or later observations could contain easier items.

The paper addresses this with a Rasch-style psychometric model. The probability that child $i$ produces word $j$ depends on the difference between the child’s latent vocabulary ability and the word’s estimated difficulty:

Development then enters through the latent ability term:

Here, $\xi_i$ represents baseline learning efficiency and $\kappa_i$ represents acceleration. This separation matters: the model is designed to distinguish a child who starts from a different baseline from one whose learning efficiency changes more rapidly with development.

Across all five datasets, the full accelerating model fits the longitudinal vocabulary responses better than the alternatives. Relative to the next-best model, item-level leave-one-out expected log predictive density improves by more than 760 in every dataset. Population estimates of $\kappa$ range from about 10.6 to 13.3, and the fitted distributions place 97.5% to 100% of children above the $\kappa=1$ benchmark, depending on dataset.

This is the paper’s main human evidence, not a vocabulary-spurt claim. The model represents continuous acceleration in latent learning dynamics.

Prediction tests whether acceleration is doing real work

A flexible developmental curve can fit historical observations and still add little predictive value. The paper therefore tests whether acceleration improves forecasts of later vocabulary measurements.

In a Norwegian subset with relatively dense longitudinal observations, a shared population-acceleration model is compared with the same model constrained to $\kappa=1$. With five previous CDI administrations available, the accelerating model improves held-out ELPD by 248.10 per child and predicts better for 92% of children over the next three months.

This prospective test is best read as predictive validation of the acceleration term. It does not identify why learning accelerates.

The supplementary analyses serve similar supporting roles rather than introducing separate theses:

The matched estimator is particularly valuable. It reduces the chance that the headline difference is produced by incompatible measurement procedures.

The language models improve, but their return curve stays almost flat

The paper trains GPT-2-small models on child-directed CHILDES data and analyzes learning across checkpoints and increasing data budgets. It also examines matched BabyLM and ClimbMix subsets.

The models learn the target words. The disagreement with children is about acceleration.

Their scaling behavior follows the familiar power-law form:

with $\beta\approx0.32$ and $R^2=0.998$ for the fitted comparison. Under this form, proportional return on additional input is constant:

In plain language, each multiplicative increase in data removes approximately the same proportion of the remaining reducible loss.

For the fitted child trajectories, proportional return instead rises with development. This gives the paper a more useful bridge than simply comparing vocabulary counts with model loss: it compares how the marginal productivity of experience changes.

That distinction reframes data efficiency. A system can scale predictably with more data while showing no evidence that it is becoming better at extracting value from later examples.

For AI teams, marginal return becomes a design metric

The paper directly establishes a descriptive child–LM difference. The engineering consequences require inference.

For model-development teams deciding how to spend a fixed training budget, one practical extension is to track learning-return curves rather than only endpoint performance. If two training strategies reach similar final loss but one produces rising marginal returns over the course of training, they may embody meaningfully different learning dynamics.

This diagnostic could be useful when comparing adaptive curricula, memory systems, meta-learning procedures, interactive training, or other approaches intended to reuse accumulated knowledge more effectively. The relevant decision is not merely whether the intervention lowers loss. It is whether later data becomes more productive under the intervention.

For product teams operating in data-constrained environments, the same logic could help distinguish strategies that need continual data expansion from strategies that improve the value extracted from a limited stream of new observations.

For benchmarking and governance teams, the result also cautions against treating human-like endpoint performance as evidence of human-like learning. Similar outputs can emerge from substantially different learning trajectories.

None of these applications requires assuming that children provide an architecture blueprint. They require only treating changing marginal return as something worth measuring.

Acceleration is established more strongly than its cause

The study proposes two broad explanations for children’s increasing efficiency. Developmental maturation could improve processing speed, memory or other learning capacities. Alternatively, children may “learn to learn”: accumulated lexical, syntactic and pragmatic knowledge could make later linguistic input more informative.

The data do not separate these mechanisms.

The child measure is caregiver-reported productive vocabulary, so changes can reflect broader production ability as well as lexical knowledge. Individual acceleration is also harder to estimate reliably because most CDI trajectories contain relatively few repeated observations. The human evidence covers monolingual, typically developing children in three languages, and the artificial-system comparison covers conventional pretrained architectures and training regimes rather than every possible learning system.

The strongest conclusion is accordingly specific: in these longitudinal child datasets, learning efficiency rises sharply with development, while the evaluated language models remain close to constant proportional returns.

For AI R&D, that is already a demanding target. The next step is not to imitate a hypothesized child mechanism prematurely. It is to build training systems whose later experience demonstrably becomes more valuable—and to measure that change directly.

Cognaptus: Automate the Present, Incubate the Future.