Divide & Verify: When Decomposition Finally Learns to Behave

A report is only as trustworthy as the sentence nobody checked.

That sounds melodramatic until an LLM-generated due diligence note, policy memo, customer support answer, or compliance summary contains three correct facts and one quiet falsehood in the same paragraph. The usual fix is simple in theory: split the answer into smaller claims, retrieve evidence for each claim, let a verifier judge them, and aggregate the results.

Simple, modular, boardroom-friendly. Naturally, it breaks in the middle.

The paper Distill and Align Decomposition for Enhanced Claim Verification introduces DAD, a framework for training the decomposer inside a decompose-then-verify factuality pipeline.¹ The important move is not merely “use reinforcement learning.” That would be the kind of sentence that sounds technical while explaining almost nothing. The useful idea is more specific: the decomposer should be trained to produce claims that are simultaneously human-readable, retrieval-ready, and aligned with the downstream verifier’s preferred granularity.

That is the mechanism worth paying attention to. In factuality pipelines, decomposition is not a preprocessing convenience. It is the interface contract between language and evidence.

The mistake is assuming smaller claims are always better

A common intuition says that fact checking improves when we split text into maximally atomic claims. One sentence becomes ten facts. Ten facts become ten retrieval calls. Ten retrieval calls produce ten verifier decisions. More granularity, more control.

Sometimes, yes. Often, no.

The paper starts from the uncomfortable observation that decomposition can create its own errors. A decomposer can over-fragment a sentence into trivial fragments, lose context, duplicate claims, or produce facts that are technically checkable but operationally useless. The opposite failure is just as dangerous: a decomposer can leave several propositions fused together, allowing one supported fragment to hide an unsupported one.

The authors formalize five desiderata for useful decomposition:

The key phrase is atomicity alignment. The right claim is not necessarily the smallest grammatical fact. It is the smallest useful factual unit for the retrieval-verifier pair being used.

That distinction matters. If a decomposer generates twenty near-duplicate fragments, the system pays for twenty retrieval and verification operations and may still get a worse answer. If it generates one coarse blob, the verifier may approve a sentence because most of it is supported. Neither is a sign of intelligence. It is just bad plumbing with better branding.

DAD trains the decomposer as the pipeline’s control surface

DAD has three parts: sequential reasoning, teacher distillation, and multi-objective reinforcement learning.

The first part reframes decomposition as a four-step reasoning process inside a single model call per sentence. The model is asked to:

1. detect whether the target sentence contains verifiable content;
2. decontextualize the sentence using nearby context;
3. identify relationships such as attribution, causality, temporality, comparison, negation, expansion, and membership;
4. extract final subclaims.

This matters because many decomposition errors are not caused by a lack of vocabulary. They come from mishandled relationships. A sentence such as “X worked with A, B, and C after leaving Y” is not just a bag of entities. It carries temporal structure, membership structure, and sometimes attribution. Split it carelessly and the verifier checks a sentence the model never actually claimed.

The second part is supervised warm-up. The authors use Llama-3.1-405B-Instruct as a teacher to generate synthetic decomposition exemplars, then fine-tune a Llama-3.1-8B-Instruct student. This gives the smaller model basic decomposition behavior and output discipline before reinforcement learning begins. Cold-start RL on structured language tasks is a fine way to manufacture chaos, so this step is not decorative.

The third part is the real alignment mechanism. DAD uses Group Relative Policy Optimization with a reward made of three terms:

The format reward makes the output parseable. Boring, essential, and therefore usually underrated.

The verifier reward aligns the decomposer with downstream verification performance. The authors compare a sparse accuracy reward with a dense Brier-score reward. The dense version uses verifier confidence rather than only binary correctness, giving the policy a smoother learning signal.

The checklist reward evaluates decomposition quality directly. It asks whether subclaims are complete verifiable propositions, retrieval-relevant, sufficiently qualified, explicit about references, and free of ungrounded additions.

This is the central lesson: verifier alignment alone is not enough. Quality checklists alone are not enough. Format compliance alone is definitely not enough, unless one’s ambition is to parse wrong answers beautifully.

DAD works because it trains the decomposer against all three pressures at once.

The main benchmark shows a better cost-quality balance, not a clean sweep

The headline result is strong but should be read carefully. Across six evaluation settings, the trained 8B DAD decomposer achieves the best aggregate performance: 71.37% balanced accuracy and 71.75% macro-F1.

The settings cover sentence-level and response-level evaluations. The sentence-level tests use ChatGPT and PerplexityAI responses to FActScore biography prompts, with Atomicity-1 and Atomicity-2 granularities and Wikipedia as the knowledge source. The response-level tests use FELM and BINGCHAT, with Google Search evidence through SerpAPI.

DAD does not win every individual column. Larger prompted models lead on ChatGPT Atomicity-1. Other prompt baselines lead in parts of FELM and BINGCHAT. This is important because the paper is not proving that DAD is universally dominant under all retrieval conditions. It is showing that trained decomposition produces the best aggregate balance across varied settings.

The operational comparison is more revealing than the single score.

Two comparisons matter most.

First, DAD beats FActScore by 6.24 macro-F1 points while generating roughly one-third as many subclaims. This directly challenges the “split everything” instinct. More subclaims are not free. Each subclaim usually means another retrieval operation and another verifier call. Worse, redundant and trivial subclaims can contaminate the verification process.

Second, DAD beats VeriScore by 1.99 macro-F1 points while using a similar number of subclaims. That suggests the gain is not simply a byproduct of producing more or fewer claims. The trained decomposer is producing claims that are more useful for the verifier.

This is the difference between volume and structure. Businesses already have enough volume. That is how they got the hallucination problem in the first place.

The DyDecomp comparison shows why verifier-only optimization is brittle

The comparison with DyDecomp is particularly useful because it exposes a subtle failure mode in reinforcement learning for verification.

DyDecomp uses verifier feedback to decide when to decompose. In DAD’s experiments, it produces only 1.66 subclaims on average and reaches 65.91% overall macro-F1. On BINGCHAT, DyDecomp has a higher F1 than DAD, but the paper reports a severe class imbalance: very high recall on SUPPORTED responses and extremely low recall on NOT SUPPORTED responses.

That asymmetry is not a minor metric footnote. It is the kind of problem that matters most in assurance workflows.

Unsupported responses often contain a mixture of true and false statements. If the decomposer leaves the response too coarse, the verifier may find enough supporting evidence for the true parts and miss the false part. The system then becomes biased toward acceptance. This is a charming property if one’s business model is rubber-stamping. Less charming if the task is compliance, risk review, or legal-adjacent analysis.

The paper’s ablation study reinforces the point. RL-only configurations using verifier rewards produce fewer subclaims. Dense Brier rewards perform better than sparse rewards, but verifier-only optimization still weakens out-of-domain performance on FELM relative to the prompt-only baseline. The best configuration combines SFT warm-up, verifier reward, and checklist reward.

Here the purpose of the ablation is not to create a second thesis. It tests whether the components actually do different jobs:

This is a good example of how to read experimental sections without turning every table into a billboard. The ablations show that reward composition matters. They do not show that the chosen recipe is the final form of factuality verification.

Dense rewards help, but the checklist keeps the model civilized

The dense verifier reward uses the Brier score, which penalizes miscalibrated confidence rather than merely checking whether a binary prediction matches the label. In the paper’s training curves, the dense reward produces smoother and more sample-efficient learning than the sparse reward. On the ablation datasets, the dense verifier reward outperforms the sparse one: 72.71% versus 70.86% macro-F1 on PerplexityAI, and 64.94% versus 58.94% on FELM.

That sounds like the answer is “use dense verifier reward.” It is not.

The same ablation also shows that optimizing only for verifier feedback can bias the decomposer toward coarser claims. Coarser claims may fit the verifier’s behavior on the training distribution, but they reduce generalization. The checklist reward counteracts this by preserving decomposition qualities the verifier signal may not properly value: explicit references, sufficient qualifiers, retrieval relevance, and absence of ungrounded additions.

This is where DAD becomes interesting for business AI systems. The checklist is not just a technical scoring device. It is an operational policy written as a training signal.

A compliance team may care that a subclaim includes time, jurisdiction, source attribution, and the exact entity name. A customer support team may care that a claim avoids implied promises. A research workflow may care that a claim separates data from interpretation. Those are not generic “accuracy” preferences. They are interface requirements.

Reward design, in this context, is governance design. Unfortunately, it is also more work than writing “be accurate” in a system prompt. Civilization has costs.

Human evaluation confirms that DAD is not just gaming the verifier

The authors also conduct human evaluation on 429 sentences drawn from 25 model responses. Three expert annotators, blinded to the decomposition method, judge outputs across completeness, uniqueness, coherence, verifiability, and clarity. Agreement is generally strong, with three-way agreement between 74% and 93% and Fleiss’ kappa between 0.5 and 0.88.

DAD scores strongly across the five dimensions:

The most revealing contrast is FActScore. It achieves 0.88 completeness, matching DAD on that dimension, but only 0.04 uniqueness. In other words, it covers the sentence by producing overlapping fragments. That can look safe if one only asks, “Did we miss anything?” It looks less safe when one pays for retrieval calls, verifier calls, and error propagation.

The appendix examples make the issue concrete. In one Harrison Ford example, FActScore produces many date fragments such as “1942 is a year” and “July is a month.” These are technically factual, emotionally harmless, and operationally silly. VeriScore makes a different error in the same example, mislabeling Dorothy as Ford’s father. DAD correctly separates date, birthplace, mother, and father.

The point is not that DAD will never make mistakes. The point is that decomposition quality has several dimensions, and optimizing only for coverage can create junk facts. Junk facts are still junk even when wrapped in an evaluation pipeline.

The scale result is a quiet warning to brute-force AI procurement

One of the more commercially relevant findings is that the trained 8B DAD model is competitive with much larger prompted models. The 8B DAD model reaches 71.75% overall macro-F1. Prompted Llama-3.3-70B reaches 71.30%. Prompted Llama-3.1-405B reaches 71.07%. Scaling from 70B to 405B under the same prompt gives only a 0.23-point F1 improvement, while training the 8B model improves 2.58 points over its prompt-only version.

This should be read with restraint. The result does not prove that small models beat large models generally. It shows that, for this decomposition role inside this verification pipeline, task-specific training can beat brute-force prompting.

That is still a useful lesson. Many enterprise AI systems are not bottlenecked by a lack of raw model size. They are bottlenecked by poorly trained interfaces between modules: the retriever does not receive the right query, the verifier receives the wrong claim, the summarizer receives evidence at the wrong granularity, and the final report aggregates confidence as if all upstream steps were sane.

When the problem is interface alignment, scale is an expensive substitute for design.

The business value is cheaper diagnosis, not automatic truth

For enterprises, the immediate value of this paper is not “now we can verify everything.” That would be adorable. The value is more practical: factuality pipelines can become more diagnosable and more cost-controlled when decomposition is trained as a first-class component.

A useful implementation pathway looks like this:

The direct paper result is that DAD improves aggregate verification performance across the tested settings while maintaining moderate subclaim granularity.

The business inference is that companies should stop treating claim splitting as a prompt template hidden inside the evaluation stack. It should be a trained, monitored, and domain-adapted layer.

What remains uncertain is how far the method transfers to specialized domains such as finance, law, healthcare, engineering documentation, or multilingual enterprise knowledge bases. The paper’s tests are mainly English, use Wikipedia and Google Search evidence, and do not include multi-hop verification benchmarks. That is a serious boundary, not a ceremonial limitation paragraph inserted to appease reviewers.

The non-aligned verifier test is promising, but not a free portability pass

The authors also test DAD with ClearCheck-8B, a verifier not used as the main aligned verifier. DAD paired with ClearCheck generally outperforms a Llama-8B prompt baseline paired with ClearCheck, suggesting that DAD’s decomposition quality has value beyond direct alignment with Bespoke-MiniCheck-7B.

But the result is mixed. On BINGCHAT, ClearCheck performance drops sharply, and the paper attributes this partly to long claims and ClearCheck’s high-confidence outputs tending to over-predict NOT SUPPORTED. This is the correct level of optimism: DAD’s decompositions are useful beyond the original verifier, but verifier-specific optimization still matters.

For deployment, that means a trained decomposer should be evaluated with the actual verifier, retriever, knowledge source, and domain texts used in production. “It worked with a verifier in a paper” is not a validation plan. It is a conversation starter wearing a lab coat.

The remaining errors show where verification still leaks

The appendix error analysis is useful because it prevents the wrong conclusion. DAD improves decomposition, but the pipeline still fails when retrieval and verification fail.

Three error patterns stand out.

First, decomposition can still be too coarse. In one example, the claim that Howard Taylor attended the Royal College of Surgeons and received honors is treated as a single unit. Evidence supports fellowship and honors, but not attendance. The verifier accepts the composite claim anyway. This is exactly the kind of failure decomposition is supposed to prevent, which makes it a useful reminder that “improved” does not mean “solved.”

Second, verifiers may lack world knowledge. In the Mauro Icardi example, a generated claim says he was on loan from the Argentina national team. Evidence supports the club loan from PSG, but a national team loan is not a normal football concept. The verifier does not catch the impossibility.

Third, evidence can be stale or underspecified. The Eric Hacker example involves a time-sensitive free-agent claim as of March 2023. Evidence supports free-agent status in other periods, but not the stated date.

These failures matter for enterprise use because many business claims are time-sensitive, jurisdiction-sensitive, or concept-sensitive. A model checking “free agent as of March 2023” has the same structural problem as one checking “policy active as of Q2,” “counterparty licensed in Singapore,” or “supplier certified under the latest standard.” The date is not decoration. It is part of the claim.

What system builders should copy from DAD

The most transferable part of the paper is not the exact model stack. It is the design pattern.

First, define the claim interface. Decide what counts as a useful subclaim for the downstream verifier. This includes entity resolution, qualifier preservation, relationship handling, and acceptable granularity.

Second, measure subclaim count as an operational variable. It is a proxy for retrieval cost, verifier cost, latency, and noise exposure. A factuality system that doubles subclaim count should justify the extra cost with better accuracy or better diagnostic value.

Third, train the decomposer against both downstream performance and independent quality checks. A verifier reward teaches the model what works for the verifier. A checklist reward keeps it from exploiting verifier quirks in ways that damage generalization.

Fourth, evaluate by failure class, not only aggregate F1. Unsupported-claim recall, stale-evidence failures, ambiguity failures, and verifier-reasoning failures are different operational risks. Combining them into one score is tidy. Tidy is not the same as useful.

Finally, keep the human review surface visible. DAD’s structured subclaims are easier to inspect than raw paragraphs. That matters because in high-stakes settings, verification systems should support review, not impersonate final authority.

Boundaries before anyone gets too excited

DAD is a meaningful advance, but the boundary conditions are clear.

The main results optimize around a specific verifier, Bespoke-MiniCheck-7B. The ClearCheck experiment helps, but does not establish broad verifier portability. The SFT stage depends on synthetic decompositions from a single 405B teacher, which may shape the student’s decomposition style. The checklist reward depends on an LLM judge, which introduces another model-mediated evaluation layer.

The evaluation is primarily English. The evidence sources are Wikipedia and Google Search. The experiments do not cover multi-hop verification benchmarks. Human evaluation is valuable but limited in scale, and annotators are co-authors, even though they are blinded to method identity.

The sentence-level segmentation design is also a practical constraint. DAD uses local context around a target sentence, which is efficient and fair for the benchmark comparison. But some business claims span multiple paragraphs, attachments, tables, or policy documents. A sentence-level decomposer may miss relationships that live outside the local window.

None of this weakens the paper’s core contribution. It simply keeps the contribution where it belongs: DAD improves the decomposer inside post-hoc factuality verification pipelines; it does not make retrieval perfect, verifiers omniscient, or enterprise knowledge bases magically current. We are still in the physical universe, unfortunately.

The real lesson: train the interface, not just the judge

The strongest idea in this paper is that factuality verification depends on the interface between claims and evidence. The verifier is the judge, but the decomposer writes the case file. If the case file is bloated, incomplete, ambiguous, or misaligned with the judge’s reasoning habits, the verdict suffers.

DAD shows that this interface can be trained. Sequential reasoning gives the decomposer a disciplined procedure. Teacher distillation gives it a stable starting point. Multi-objective reinforcement learning aligns it with both verifier performance and decomposition quality. The result is not merely a better score. It is a better-shaped pipeline.

For business AI, that is the more durable lesson. Reliable automation will not come from asking one giant model to “be factual” with sufficient sincerity. It will come from systems where intermediate representations are designed, trained, tested, and governed.

Decomposition finally learning to behave is a small technical victory. It is also a useful reminder: in AI systems, the boring interface is often where the intelligence either becomes operational—or quietly leaks out.

Sources

Cognaptus: Automate the Present, Incubate the Future.