Who Was Where When? AI Tries to Remember History

Archive work has a very simple-looking question at its center: who was where, and when?

That question looks harmless until a machine has to answer it from a century-old newspaper, after OCR has mangled the spelling, the place names have shifted, the language is not always English, and the text only implies the answer through an event, job title, or institutional affiliation. At that point, “extracting information” becomes less like copying a fact from a sentence and more like making a legally cautious inference from a witness who speaks in fragments.

That is the useful trick inside HIPE-2026, a CLEF evaluation lab for person–place relation extraction from multilingual historical texts.¹ The paper is not mainly interesting because it introduces another benchmark. The world is not suffering from a shortage of benchmarks. It is interesting because it forces AI systems to separate three things that business document systems usually blur together: textual evidence, temporal scope, and the cost of inference.

In other words, HIPE-2026 asks a model not only whether a person is linked to a place, but what kind of link the document actually supports. That distinction is small on paper and large in production. Conveniently, small distinctions are where automated systems often become expensive.

The task is not “find two names in the same article”

The tempting reading of person–place extraction is simple: detect a person, detect a place, and connect them if they appear nearby. That is the co-occurrence trap.

HIPE-2026 is designed against exactly that shortcut. The paper defines the task as relation classification over candidate person–place pairs inside a document. For every person–place pair, the system must decide whether the text supports a relation between the person and the place. The key point is that support may be explicit, indirect, temporally bounded, or absent.

The benchmark defines two related labels:

The distinction is doing most of the intellectual work.

A person may have been connected to a place historically without being there at publication time. A person may be institutionally affiliated with a location without the article proving physical presence. A person may be inferred to have been at an event location because they gave a speech at that event, even if the article never says, “he was standing in Clear Pond.” The model must decide how much inference is warranted, and then stop before it starts inventing geography.

That final discipline matters. It is easy for an LLM to be “reasonable” in a way that is evidentially reckless. HIPE-2026 makes that recklessness visible.

at and isAt turn relation extraction into temporal reasoning

The paper’s mechanism is worth slowing down for. The at label is right-bounded by publication date. It asks whether the document supports the claim that the person was present at a place at any point before publication. The isAt label is narrower. It asks whether the person was there around the publication moment.

So isAt is not a separate relation floating beside at. It is a temporal refinement of it.

That creates an epistemic dependency. If a system predicts isAt = +, it is effectively claiming that some version of at must also hold. The paper notes that inconsistent predictions such as at = false and isAt = + are practically permitted, but they are conceptually wrong. This is a neat benchmark choice: the evaluation can expose whether a system merely classifies labels independently or maintains a coherent internal interpretation.

For business AI, that difference is not academic. Many document-intelligence products already claim to extract relationships from contracts, filings, media archives, clinical notes, or compliance records. The hard part is often not entity recognition. It is knowing whether the relationship is historical, current, inferred, contradicted, or merely adjacent in text.

HIPE-2026 compresses that entire problem into one deceptively narrow question: did the document actually give us enough evidence to put this person at this place?

The “probable” label is controlled inference, not polite uncertainty

The most interesting label is not true or false. It is probable.

At first glance, probable may look like a soft middle category for weak evidence. That undersells it. The paper grounds this label in abductive interpretation: the idea that understanding language often means identifying the assumptions needed to make discourse coherent. A text may imply a relation through event participation, institutional role, or narrative structure without stating it directly.

This is where the benchmark becomes more than extraction.

A system must distinguish among four cases that ordinary pipelines often collapse:

The paper’s example from the Tabor City Tribune makes this concrete. Col. Gruenwald is linked to Myrtle Beach Air Force Base through his role as commanding officer, to Clear Pond through an event where he delivered an address, and not to nearby places such as Conway or Loris merely because they appear in the same article. That is exactly the kind of distinction a naïve “entities in the same paragraph” system would cheerfully flatten. Machines do enjoy being overconfident interns.

The probable label therefore has a practical function. It lets the benchmark reward disciplined inference without rewarding hallucination. That is a valuable design pattern for enterprise AI: do not force every relation into binary truth when the evidence itself is graded.

The data design tests noise, language, and domain shift separately

HIPE-2026 uses two evaluation sets.

Test Set A contains historical newspaper articles in French, German, English, and Luxembourgish, spanning roughly the nineteenth and twentieth centuries. These documents come from the HIPE-2022 data and carry the expected messiness of digitized archives: OCR errors, historical spelling variation, multilinguality, and weakly structured narrative.

Surprise Test Set B consists of French literary texts from the sixteenth to eighteenth centuries. This set evaluates domain generalization, and only the at relation is assessed there.

That division is important. The paper is not simply asking whether models can perform relation extraction on one dataset. It separates the main task from a domain-shift stress test. Newspapers and literary texts encode people and places differently. A system that learns newspaper habits may fail when the genre changes. This is not a minor issue for business deployment. Models trained on one document family often behave well in demos and then quietly degrade when the document source changes. The spreadsheet looks fine until reality imports a new column.

The pilot study gives an early sense of difficulty. The authors report annotations over 119 person–place pairs by three independent annotators. Agreement for at ranges from 0.7 to 0.9 Cohen’s kappa, while isAt ranges from 0.4 to 0.9. The pattern is unsurprising but useful: historical presence is easier to agree on than immediate temporal presence. GPT-4o reportedly reaches up to 0.8 agreement with the gold standard for at, while isAt is lower and more variable, between 0.2 and 0.7.

These are pilot numbers, not final leaderboard results. Their purpose is diagnostic. They show that the task is feasible, that humans can annotate it with moderate to high consistency, and that current frontier models may already align reasonably well on the broader at relation. They also show where the difficulty lives: temporally constrained inference.

The candidate-pair explosion makes efficiency part of the task

The paper’s most business-relevant move is not the historical corpus. It is the evaluation design.

HIPE-2026 uses three evaluation profiles:

The accuracy profile uses macro-averaged recall, also called balanced accuracy. This matters because relation labels may be imbalanced. A model should not get an impressive score simply by doing well on common labels while failing rare ones. For Test Set A, macro recall is computed separately for at and isAt, then averaged for ranking.

The accuracy–efficiency profile is where the benchmark becomes especially modern. The authors explicitly motivate it with the cost of very large models and the scale of digitized historical material. If a document has $p$ persons and $l$ locations, the candidate relation space grows as:

That multiplication is harmless in a short example and brutal in a national archive. Every additional entity type increases the number of pairwise decisions. Every pair may require context reading, temporal interpretation, and possibly explanation. If each decision is handed to a frontier LLM with a long prompt, the invoice becomes a historical document too.

The benchmark therefore asks not only which system is most accurate, but which system is accurate enough under resource constraints. Parameter count, model size, and accuracy are integrated into an efficiency-aware ranking profile. This is the part AI builders should notice. The market is slowly learning that “best model” and “best deployed system” are different objects.

What the paper directly shows, and what Cognaptus infers

The paper directly shows three things.

First, it formalizes person–place relation extraction in historical texts as a temporally scoped classification task. This is the core contribution. The at and isAt design turns a vague “where was this person?” question into an annotation scheme that can be evaluated.

Second, it extends the HIPE line of shared tasks from named entity recognition and linking into semantic relation extraction. That matters because knowledge graphs need edges, not just nodes. A system that recognizes “Col. Gruenwald” and “Clear Pond” has not yet built historical knowledge. It has only collected ingredients.

Third, it proposes an evaluation setup that treats accuracy, efficiency, and generalization as separate but related concerns. This is a more production-aware view of AI evaluation than accuracy alone.

Cognaptus’ business inference is narrower and more practical: this benchmark is a useful proxy for document-intelligence systems that need to build reliable knowledge graphs from imperfect text. The strongest application pathway is not generic chat. It is structured extraction for archives, media intelligence, historical research platforms, legal discovery, compliance investigation, and enterprise knowledge-base enrichment.

The pattern transfers because many business documents have the same hidden structure:

This does not mean a HIPE-2026 system can be dropped into a compliance workflow tomorrow. It means the benchmark isolates a class of reasoning that many enterprise systems need and often pretend is solved.

The benchmark rewards architecture, not just prompting

One likely outcome of HIPE-2026 is that simple prompting will look attractive in the early stages. Large language models are good at reading context, explaining decisions, and making plausible abductive inferences. The pilot evidence suggests that GPT-4o can align well with human judgments for at.

But the task design quietly encourages more structured architectures.

A robust system may need several layers:

1. Entity normalization to handle noisy historical mentions.
2. Candidate-pair generation to enumerate person–place combinations.
3. Evidence retrieval to select relevant document spans.
4. Temporal validation to distinguish at from isAt.
5. Consistency checking to catch impossible label combinations.
6. Efficiency controls to avoid sending every weak candidate to the most expensive model.

The best production architecture may therefore be hybrid. A smaller model or rule layer can eliminate obvious negatives. A stronger model can handle ambiguous cases. A final validation layer can enforce temporal consistency and produce audit-friendly explanations.

That architecture is less glamorous than “just ask the LLM.” It is also more likely to survive contact with a real archive.

Where the evidence should not be overread

HIPE-2026 is a benchmark proposal and task description, not a completed production study. That boundary matters.

The pilot study is useful but small: 119 annotated person–place pairs from English and French development data. The reported agreement ranges show feasibility and difficulty, not final model superiority. The GPT-4o results are promising, especially for at, but the variability on isAt warns against assuming that frontier models already solve temporal relation extraction.

The efficiency profile is also a design commitment rather than an operational cost model. It asks teams to report model size and parameter count, and it incorporates these into a robust ranking metric. That is valuable, but real deployment cost also depends on batching, retrieval strategy, hardware, prompt length, caching, latency constraints, and human review requirements.

Finally, the domain-generalization test is meaningful but specific. Early modern French literature is a useful surprise domain, but it is not the universe of historical text. A system that handles newspapers and literary French may still struggle with court records, colonial administrative reports, handwritten correspondence, or mixed-language trade archives.

So the practical conclusion is not “HIPE-2026 solves historical knowledge extraction.” It is more modest and more useful: HIPE-2026 gives the field a sharper way to measure whether AI systems can infer person–place relations responsibly under noisy, multilingual, temporally constrained conditions.

The business value is evidential discipline at scale

The durable lesson from this paper is that extraction quality depends on restraint.

An enterprise knowledge graph is not improved by adding more edges if those edges blur explicit facts, plausible inferences, and unsupported guesses. In many business settings, that distinction is the product. A compliance analyst does not only need to know that a person is associated with a location. They need to know whether the association is directly stated, inferred through an event, current at the relevant date, or merely the result of textual adjacency.

HIPE-2026 makes those distinctions measurable.

That is why the paper is useful beyond digital humanities. Historical archives are an unusually honest testbed for AI reasoning because they remove the comforts of clean text, stable language, and obvious context. If a system can handle that environment, it may be closer to the kind of document intelligence businesses actually need: multilingual, noisy, cost-aware, and careful about what the evidence permits.

The question “who was where when?” sounds simple. The paper shows why it is not.

And for AI builders, that is the point. The future of document intelligence will not be won by systems that merely recognize names. It will be won by systems that know when a relation is true, when it is probable, when it is current, and when it is just a place name standing nearby, minding its own business.

Cognaptus: Automate the Present, Incubate the Future.