Common Is Not Defining: Testing Whether Language Models Understand Category Relations
TL;DR for operators A model-review team may need to decide whether a feature is essential to a category or merely common in the data. That distinction matters because a strong association can otherwise become an unsupported ontology rule, automated policy, risk classification, or product requirement. For six embedding-based transformer models, scores initially appeared to separate defining properties from properties that were only statistically common. Once researchers controlled for human-rated prevalence—how often each property occurs—most of that separation disappeared. The same raw score that seemed to reveal conceptual structure was largely explained by frequency. GPT-4 retained a substantially stronger distinction under the same control. ...