Divide & Verify: When Decomposition Finally Learns to Behave

Opening — Why this matters now

Large language models are no longer just creative assistants. They draft policy briefs, summarize earnings calls, generate medical explanations, and produce due diligence notes. In other words: they generate liability.

As organizations integrate LLM outputs into decision-making pipelines, factual verification has shifted from academic curiosity to operational necessity. The dominant architecture—decompose, retrieve, verify, aggregate—looks elegant on paper. In practice, it behaves like a fragile supply chain. If decomposition is noisy, retrieval misfires. If atomicity is mismatched, the verifier underperforms. If granularity drifts, costs explode.

The paper “Distill and Align Decomposition for Enhanced Claim Verification” (arXiv:2602.21857v1) addresses the structural weak point: the decomposer itself. Rather than treating decomposition quality and verifier alignment as separate tuning problems, the authors argue for joint optimization. And surprisingly, it works.

Background — The silent bottleneck in decompose–then–verify

The canonical pipeline follows this structure:

1. Split a sentence into subclaims.
2. Retrieve evidence per subclaim.
3. Let a verifier judge each claim.
4. Aggregate verdicts.

Simple. Modular. Appealing.

Yet decomposition quietly governs everything downstream. The paper formalizes five desiderata:

Existing systems optimize fragments of this list. Some aim for atomic extraction (often too aggressively). Others optimize verifiability. A few use RL to decide when to decompose. Almost none jointly optimize decomposition quality and verifier behavior under a unified objective.

That fragmentation produces predictable pathologies:

• Over-decomposition inflates retrieval and inference cost.
• Under-decomposition hides unsupported fragments inside composite claims.
• Verifier misalignment penalizes otherwise reasonable decompositions.

The core thesis of this work is that decomposition and verification cannot be optimized independently. The verifier’s preferred granularity is latent. The decomposer must learn it.

Analysis — Distill, Reason, Align

The proposed framework (DAD: Distill–Align–Decompose) has three components.

1. Sequential reasoning inside a single call

Instead of directly emitting subclaims, the decomposer performs explicit staged reasoning:

1. Detect verifiable content.
2. Decontextualize.
3. Identify relationships (temporal, causal, attribution, etc.).
4. Extract minimal claims.

This structure is enforced through tagged reasoning blocks. Crucially, this remains a single model call per sentence—avoiding the latency of multi-stage pipelines.

This design is not cosmetic. It regularizes behavior. Relationship preservation, qualifier handling, and pronoun resolution become explicit intermediate constraints.

2. Teacher distillation for stable initialization

Cold-start RL on decomposition is risky. The authors warm-start an 8B student model using decompositions generated by a 405B teacher.

This serves two purposes:

• Format compliance and instruction-following competence.
• A prior on high-quality decomposition behavior.

Only after supervised fine-tuning does reinforcement learning begin.

3. Multi-objective RL with Group Relative Policy Optimization (GRPO)

Here is the structural innovation.

Instead of optimizing for verifier accuracy alone, the reward combines three normalized components:

The verifier reward is tested in both sparse (binary correctness) and dense (Brier-based) forms. The dense reward, using confidence-sensitive scoring, produces smoother learning and better sample efficiency.

GRPO (Group Relative Policy Optimization) stabilizes training by computing advantages relative to group-level reward means. This avoids the complexity of a critic network and keeps training efficient for large LLMs.

In effect, the decomposer is no longer just “helpful.” It is economically rational under a structured, multi-dimensional reward surface.

Findings — Accuracy, Granularity, and Cost

Across six evaluation settings, the trained 8B decomposer achieves 71.75% macro-F1—the best aggregate performance.

Performance Summary

Three observations matter operationally.

1. Over-decomposition is expensive and counterproductive

FActScore generates nearly 23 subclaims per input, inflating retrieval and verification calls. DAD reduces this to ~8 while improving F1 by +6.24 points.

Compute efficiency is not just elegance—it is cost containment.

2. Verifier-only RL biases toward under-decomposition

DyDecomp produces only ~1.66 subclaims. This favors the SUPPORTED class but severely harms recall for NOT SUPPORTED cases.

For compliance workflows, this imbalance is unacceptable. Missing unsupported claims is materially riskier than flagging extra ones.

3. Specialized training beats brute-force scale

Prompted 70B and 405B models barely outperform prompted 8B models. Fine-tuning the 8B with DAD yields stronger results than scaling parameters 50×.

That is a sobering reminder: alignment and structure often trump scale.

Human Evaluation — Does it actually decompose well?

The authors manually evaluated 429 sentences across five quality dimensions.

FActScore, interestingly, achieved strong completeness but near-zero uniqueness—suggesting brute-force redundancy to guarantee coverage.

Redundancy increases cost without improving epistemic quality.

DAD maintains coverage while minimizing duplication—a better cost–quality equilibrium.

Implications — What this means for AI assurance

This paper has implications beyond claim verification.

1. Modular AI systems require joint optimization

If a pipeline is multi-stage, optimizing stages independently will produce systemic misalignment. The decomposer–verifier coupling is an example of a broader systems principle.

In enterprise AI stacks—retrieval-augmented generation, compliance auditing, risk detection—the weakest upstream module dictates downstream reliability.

2. Reward design is governance design

The checklist reward formalizes normative criteria: completeness, clarity, qualifier sufficiency. Encoding these into the training objective transforms abstract evaluation guidelines into operational behavior.

That is governance by gradient.

3. Smaller, specialized models can rival giants

With structured reasoning and aligned rewards, an 8B model reaches performance comparable to 70B–405B prompted systems.

For businesses, this means:

• Lower inference cost.
• Greater deployability.
• More controllable alignment surfaces.

Scale is not a substitute for architecture.

4. Verification cost becomes predictable

Average subclaim count directly maps to:

$$ \text{Verification Cost} \propto \text{Subclaims} \times (\text{Retrieval} + \text{Verifier Calls}) $$

By stabilizing subclaim count near ~8 while maintaining coverage, DAD moves the system toward predictable operational expenditure.

For regulated industries, predictability matters as much as accuracy.

Limitations — The uncomfortable truths

The framework is tested primarily with a specific verifier and English datasets. Multi-hop reasoning and multilingual verification remain open challenges. The checklist judge is itself an LLM, introducing second-order bias.

And importantly: retrieval quality still caps performance. Decomposition can be optimal; evidence may still be stale.

No pipeline is stronger than its weakest external knowledge source.

Conclusion — When structure meets alignment

Decomposition is not a preprocessing step. It is a control surface.

By reframing claim extraction as structured reasoning and aligning it through a multi-objective reward, the authors demonstrate that verification accuracy and decomposition quality need not be traded off.

The lesson for AI system builders is simple:

If your components talk to each other, train them as if they do.

Because they already are.

Cognaptus: Automate the Present, Incubate the Future.