Unsupervised and Self-Supervised Learning
First predict the outcome, then adjust a control and inspect the numerical evidence. Both experiments are deterministic and run entirely in your browser.
Experiment 1 — PCA preserves variation
Choose a line through the sample mean of five observations. Centering and the PCA benchmark use the same data and squared reconstruction objective.
What should stay true?
The PCA direction minimizes centered squared reconstruction error. At fixed data and rank, retained variance plus lost variance equals total centered variance. The direction of a principal axis is only defined up to sign.
Experiment 2 — Raw text creates its own target
Each sentence is an unlabeled four-word record. We hide the second word and use the original word as a training or evaluation target. The toy predictor uses counts and add-one smoothing, not a neural network.
Inspect the raw training corpus
The original 15 sentences remain unchanged during evaluation. Each generates a masked-word input and an observed middle-word target.
Experiment 3 — Did the objective predict held-out targets?
All three predictors score the same five held-out middle words. Loss is the mean negative log probability assigned to the true word; lower is better. This tiny deliberately constructed example is not a generalization benchmark.