Three Loops Toward Self-Improvement

TL;DR for operators

Model teams encounter three distinct constraints after a base model exists. New proprietary knowledge may remain accessible through retrieval without becoming reliably encoded in the weights. High-quality unique training text eventually becomes scarce even when additional training compute is available. Improvements to the training recipe still depend heavily on humans proposing, implementing, and testing experiments.

Zitong Yang’s thesis, Continually self-improving AI, treats these as separate targets for self-improvement rather than one sweeping capability.¹ Its experiments report three corresponding results. Synthetic representations improve continued pretraining on small corpora. Conditional synthesis extracts additional training value from a fixed document pool. Execution-guided search improves model-training experiments by feeding measured outcomes back into subsequent idea generation.

For an operator, this changes the resource-allocation problem. Before buying more data or adding another automated agent, identify whether the limiting input is knowledge, unique training signal, or experimentation capacity. Each requires a different loop, and each comes with a different verification burden.

Self-improvement starts with identifying what is actually constrained

The term “self-improving AI” encourages a misleadingly unified mental model. The thesis instead defines three capabilities: continuing to acquire parametric knowledge after initial training, producing training signals that improve performance beyond what the available human-generated data alone provides, and autonomously improving the learning algorithms used to create models.

That decomposition matters operationally because the three problems consume different resources.

The common mechanism is not autonomous redesign. It is the conversion of structure already available to the system—relations inside a corpus, correlations across documents, or outcomes from executed experiments—into additional training or search signal.

Small corpora become more useful when knowledge is represented more than once

Consider a team with a small specialized corpus. Repeatedly training on the raw documents does not necessarily make the model absorb the knowledge efficiently, while retrieval leaves the information external to the model.

The thesis tests a method called EntiGraph for this setting. It extracts salient entities and generates analyses of relations among entity pairs and triplets, creating many synthetic views of the same underlying corpus before continued pretraining.

On QuALITY, closed-book accuracy rises from 39.49% for Llama 3 8B Base to 56.22% after training on 455 million EntiGraph tokens. Raw continued pretraining underperforms the base model, while a rephrasing baseline improves more slowly. A transfer experiment using 15 Coursera lecture transcripts also moves accuracy from 48.26% to 53.87%.

These are the main knowledge-acquisition results. Other experiments answer narrower questions. A weaker Llama 3.1 8B Instruct generator produces a similar scaling slope, serving as a robustness check against the concern that the main GPT-4-Turbo-generated corpus is simply transferring a stronger teacher’s capability. A manual review of 150 generated sentences estimates 94.4% factuality among non-subjective sentences, but the sample is small and drawn from relatively simple material.

The retrieval experiment clarifies the operating role. EntiGraph continued pretraining plus RAG reaches 62.60% accuracy versus 60.35% for the base model plus RAG, with both at 99.63% Recall@8. Parametric acquisition therefore does not need to replace retrieval. In this experiment, the two mechanisms compose.

Cognaptus inference: organizations with stable, specialized corpora should evaluate whether frequently required knowledge deserves both a retrieval path and a parametric-learning path. The business case depends on repeated usage being valuable enough to justify synthetic-data generation and continued-training cost.

Fixed data does not necessarily mean fixed learning signal

The second constraint is different. Suppose the organization has compute available but cannot keep expanding its supply of unique, high-quality documents. Repeating the same data is a strong baseline, but repetition leaves correlations among different documents largely implicit.

Synthetic Bootstrapped Pretraining (SBP) pairs semantically related documents, trains a synthesizer to predict one related document from another, generates additional synthetic text, and then jointly pretrains on real and synthetic data. The thesis interprets this as using one document to infer latent concepts and then sampling another realization of those concepts.

Under matched training-compute budgets, SBP consistently beats repeating the available unique data. On average QA accuracy, it recovers about 42% of the oracle improvement at the 200B-token scale, 48% at 1T tokens with a 3B model, and 58% at 1T with a 6B model. The oracle is allowed access to more unique data, so SBP does not eliminate the value of genuinely new information. It recovers part of that advantage using structure already present in the existing corpus.

The quality measurements prevent an overly generous reading. Non-factuality is reported at 15.1% in the 200B-scale synthetic distribution, 8.7% at 1T with the 3B model, and 6.5% with the 6B model, versus 1.8% for real data. The implementation also uses an external Qwen3-0.6B embedding model to identify related document pairs, so the reported pipeline is not completely self-contained.

For model developers approaching a data ceiling, the economic comparison is therefore not “synthetic versus real.” It is whether generating and filtering synthetic examples produces more marginal capability per unit of compute than additional repetition, data acquisition, or a larger one-shot training run.

Automated research improves when ideas are forced through execution

The third loop moves from training data to the learning process itself.

Generating plausible research ideas is insufficient. An automated research system needs to translate an idea into code, execute it safely, score the resulting model behavior, and make subsequent proposals conditional on what happened.

The thesis builds such an execution layer for nanoGPT pretraining and GRPO post-training. In the reported environments, execution-guided search raises MATH validation accuracy from 48.0% to 69.4% and reduces nanoGPT time to reach the target validation loss from 35.9 minutes to 19.7 minutes.

The human comparison is informative. The automated system slightly exceeds the reported best-human GRPO result of 68.8%, but the best human nanoGPT result reaches the target in 2.1 minutes. The experiments therefore show that automated search can produce meaningful improvements inside constrained environments, not that automated researchers have generally surpassed human model engineers.

The matched-budget comparison isolates a more useful mechanism. Evolutionary search beats best-of-N from the first feedback-informed epoch onward. Once new ideas are conditioned on prior execution outcomes, the same sampling budget becomes more productive than generating candidates independently.

A reinforcement-learning extension raises average execution reward without raising the maximum observed reward. For R&D systems, that distinction is consequential: optimizing the expected quality of proposals is not identical to discovering rare, exceptional improvements.

The operational boundary is the evaluation loop

None of these experiments establishes a model that recursively improves itself across arbitrary domains. The thesis itself treats that broader destination as unresolved.

Each reported mechanism depends on a deliberately constructed boundary. EntiGraph depends on source-corpus grounding and synthetic-data quality. SBP depends on document pairing, a synthesizer, and fixed evaluation protocols. Automated research depends on an executor that can correctly implement ideas and on protected metrics that accurately value the resulting experiments.

The research-search experiments also do not test whether ideas discovered at small scale remain effective on larger models or different datasets. Executor failures can suppress promising ideas, while poorly specified rewards can make measurable optimization diverge from the research outcome an organization actually values.

This produces a more concrete governance requirement than generic human oversight: every self-improvement loop needs an evidence contract defining what may change, what remains protected, what metric authorizes another iteration, and how transfer is tested before an improvement is promoted beyond the environment where it was discovered.

Build the loop around the bottleneck

The thesis makes “self-improvement” more operational by making it less monolithic.

A team struggling to internalize proprietary knowledge has a representation and continued-training problem. A developer with abundant compute but limited unique text has a data-efficiency problem. An R&D group overwhelmed by experiment throughput has an execution-and-search problem.

The experiments suggest that each can be improved without waiting for a system capable of unrestricted recursive self-redesign. They also show why those narrower gains should not be extrapolated casually. Every improvement reported here is produced inside a measurable loop with fixed data access, benchmark protocols, compute budgets, or execution environments.

For operators, that is the actionable unit of self-improvement: not an AI system that gets better in the abstract, but a bounded process that can generate a candidate improvement, test it against evidence, and decide whether the result deserves another iteration.

Cognaptus: Automate the Present, Incubate the Future.