Common Is Not Defining: Testing Whether Language Models Understand Category Relations

TL;DR for operators

A model-review team may need to decide whether a feature is essential to a category or merely common in the data. That distinction matters because a strong association can otherwise become an unsupported ontology rule, automated policy, risk classification, or product requirement.

For six embedding-based transformer models, scores initially appeared to separate defining properties from properties that were only statistically common. Once researchers controlled for human-rated prevalence—how often each property occurs—most of that separation disappeared. The same raw score that seemed to reveal conceptual structure was largely explained by frequency. GPT-4 retained a substantially stronger distinction under the same control.

Do not treat semantic similarity, repeated co-occurrence, or a fluent answer as proof that a model understands what defines a category. Test whether its judgment survives prevalence control, compare it with human-rated distinctions, and evaluate the specific deployed model rather than generalizing across language models.

When a common feature becomes a policy rule

A team building a product ontology, eligibility rule, maintenance classifier, or automated risk policy must decide whether a feature belongs to the definition of a category or merely appears often in historical data. That difference determines whether the feature becomes a requirement, a default assumption, an exception trigger, or no rule at all.

A model can produce a high-confidence association in both cases. Airplanes have wings, and airplanes have passengers. Both statements are familiar, but only the first follows closely from what an airplane is. Treating the second relation as equally defining would convert a common observation into a false requirement.

Hu, van Paridon, and Lupyan test whether language models can recover this difference from language statistics alone.¹ Their result is neither a general failure nor proof of understanding. It shows why evaluation must separate statistical frequency from conceptual structure.

Human judgments retain information beyond frequency

The study uses 208 generic statements about objects, people, and animals. Eighty-four retained participants rated statements on ordinary truth, truth by virtue of category membership, property prevalence, and cue validity.

A property is “principled” when it is judged true because of what the category member is, rather than simply because the property is frequently observed. Human ratings reproduced this distinction. Principled statements received higher truth ratings and a much larger advantage on by-virtue-of ratings. Those judgments still predicted relation type after prevalence was statistically controlled.

The airplane example makes the gap concrete. “Airplanes have wings” received human means of 2.9 for ordinary truth and 2.6 for by-virtue-of truth. “Airplanes have passengers” received 1.8 and 0.6. Frequency affects both, but it does not exhaust the human distinction.

This human result is the measurement anchor. Without it, a model could receive credit for reproducing frequent category-property pairings even when its score contains little information about why the property belongs with the category.

Embedding similarity mostly follows prevalence

The researchers evaluated BERT, ALBERT, DistilBERT, RoBERTa, GPT, and GPT-2 using cosine similarity between a category term and its property phrase. Principal-component removal was applied to create more contrastive, approximately decontextualized embeddings; this was an implementation step rather than a separate test.

In raw comparisons, all six models assigned higher similarity to principled relations. Read alone, that appears to show conceptual sensitivity. After controlling for human-rated prevalence, however, the association largely vanished; ALBERT remained only marginally above zero.

The finding does not mean the embeddings contain no relevant information. It means most of the apparent separation can be explained by a simpler signal: principled properties also tend to occur more often. Raw similarity therefore combines frequency with any information about category membership and cannot serve as a clean measure of the latter.

For ontology review or policy automation, this is a confounding problem. A system trained on historical text may elevate frequent features into category rules, producing invalid mandatory attributes, overbroad risk labels, brittle exceptions, or unsupported assumptions about customers and populations.

GPT-4 preserves a stronger residual distinction

GPT-3.5 and GPT-4 were tested through direct ratings rather than embedding similarity. Each model rated every by-virtue-of statement on a scale from -3 to 3, with 15 repetitions per item. The averaged responses had variance below 0.01.

After prevalence control, GPT-3.5 retained only weak predictive information: $t=2.05$, $p=.04$. GPT-4 remained strongly predictive: $t=5.18$, $p<.00001$. Item-level correlations with the human principled-versus-statistical difference were .21 to .28 for the embedding models, .49 for GPT-3.5, and .61 for GPT-4.

Together, the prevalence-controlled result and the .61 correlation indicate that GPT-4 captured substantially more than the frequency pattern in this stimulus set. They do not imply equivalence with human judgment. The bounded conclusion is that language statistics can, in principle, preserve information about category-property relations beyond simple prevalence.

The comparison also shows why capability claims must be model-specific. GPT-3.5 and GPT-4 did not perform alike, and the embedding models showed a different pattern. Evidence should attach to a tested model, probe, item set, and control condition—not to “language models” as a class.

A prevalence-controlled review protocol

The following framework is a Cognaptus inference from the study, not a procedure validated by the paper.

This protocol is most relevant when a user must decide whether an attribute is required, causal, defining, or merely typical. It can improve ontology construction, catalog rules, eligibility checks, compliance logic, and model-governance reviews. Prevalence control improves the validity of the test; it does not reveal the model’s internal representation.

What the evidence leaves unresolved

The study is behavioral and comparative. It does not identify why GPT-4 succeeds. Training data, scale, and post-training details are unavailable, and no intervention isolates the responsible mechanism. The result could reflect a richer world model, linguistic regularities that support the distinction, post-training behavior, or a combination.

The probes also differ. Older transformer models were assessed through embedding cosine similarity, while GPT-3.5 and GPT-4 produced direct ratings. The study therefore compares measured outputs, not architectures under one uniform instrument.

Generalization is limited to 208 selected English generics and a United States sample of native English speakers. Other languages, technical taxonomies, social categories, and operational domains may introduce different confounds. Behavioral alignment also does not demonstrate a human-like causal representation or show that people learn the distinction from language alone.

Remove the easy signal before claiming understanding

A strong category-property association should not be accepted as evidence that a model knows what is essential to the category. Before approving an ontology rule, classifier, or automated policy, reviewers should test whether the apparent understanding survives after prevalence is removed.

GPT-4’s result shows that richer relational sensitivity is possible. The embedding-model results show that it is not automatic. The appropriate standard is controlled, model-specific evidence: distinguish what is common from what follows from category membership, verify the pattern against human judgments, and keep mechanism claims outside the evidence boundary.

Cognaptus: Automate the Present, Incubate the Future.