Make the Structure Survive: Forma Turns Synthetic Clinical Cases Into Auditable Outputs

TL;DR for operators

If synthetic cases are going to support training or evaluation, surface plausibility is a weak control objective. The harder requirement is preserving the relationships that make each case meaningful.

Forma tests that idea by specifying a person-specific psychological structure before generation and asking whether those directional relationships can be recovered afterward. In the full condition, an external probe reaches MCC +0.41 and AUC 0.70 for directed-edge recovery. When the structural formulation is removed, performance falls close to chance: MCC +0.03/AUC 0.52 with demographics and self-report still present, and +0.01/0.50 under zero-shot generation.

The operational pattern is broader than clinical text: represent the structure that must survive into an output, generate against that specification, and audit the realized output against it. This can improve control over synthetic benchmarks and training stimuli. It does not establish that the generated cases resemble real patient populations or are appropriate for patient-facing clinical systems.

Plausible prose can preserve the facts and lose the case

Imagine generating several synthetic cases for the same training exercise. Each needs the same visible ingredients, but more importantly, it must preserve why one factor leads to another. A case where an intrusive memory triggers perceived current threat is not equivalent to one that merely mentions both.

That distinction is easy to miss when evaluation focuses on fluency, symptom coverage, or whether several reviewers agree that the text sounds reasonable. A generator can include the right concepts while changing the relationships among them.

Amit Oren, Nimrod Hertz-Palmor, Dean Ariel, and Guy Laban address this problem in Generating Clinical Vignettes that Preserve Cognitive Formulations.¹ Their Forma framework treats a cognitive formulation—a person-specific account of active psychological processes and their relationships—as an explicit generation specification rather than background prose.

Forma turns the formulation into a generation contract

The implementation uses the Ehlers-Clark model of PTSD. Five psychological components become nodes in a directed graph, while directed edges specify which component influences another. Active edges also receive weights that instruct the generator how prominently the relationship should appear.

Each synthetic persona combines demographics, sampled self-report items, and a sampled graph. The full condition sends all three to the generator and then runs a validator that checks whether required components and directed links appear. Missing elements can trigger targeted patches, with up to five retries.

The strongest comparison is not zero-shot generation. It is the no-formulation condition, which keeps demographics, self-report content, and validation while removing the graph. That isolates what the explicit relational specification contributes beyond simply giving the model more clinical information.

The corpus contains 16,500 vignettes across 500 personas, 11 models, and three conditions. One model, gpt-4o-mini, is excluded from analyses requiring successful structural validation because it failed to comply reliably with the patch-marker control flow. That failure is an implementation detail with practical significance: structural prompting only helps if the selected model can also execute the surrounding protocol.

The graph leaves a recoverable trace in the finished text

The central test is edge recovery: can a reviewer infer the directional relationships that were specified before generation by reading only the final vignette?

This is main evidence, not a stylistic side analysis. The full condition is the only one where the intended relational structure is recovered materially above chance.

The paper then repeats the test with blinded human annotators on 150 vignettes. Two clinical raters reach MCC +0.41 and +0.32 in the full condition, compared with values near zero in both ablations. Their AUCs are 0.69 and 0.65. Two LLM annotators reproduce the same ordering.

That replication matters because it removes an easy explanation: the apparent structure is not merely one automated evaluator recognizing artifacts left by another model.

The edge weights themselves are less convincing as a fine-grained control dial. Forma is stronger at determining whether a relationship appears than at guaranteeing precisely calibrated narrative prominence. For an operator, that suggests separating binary structural obligations from more delicate requirements about emphasis.

Explicit structure also reduces model-to-model drift

Structural fidelity would be narrow if it changed only an edge-recovery score. The paper finds effects elsewhere in the generated cases.

Across ten retained models, vignettes for the same persona become more similar when the formulation is supplied. With MPNet embeddings, mean within-persona cosine similarity rises from 0.737 in zero-shot to 0.792 in the full condition, with paired $d_z=1.52$. Using BGE instead produces the same ordering: 0.800 versus 0.837, with $d_z=1.61$.

That encoder replication is best read as a robustness test. The relevant result is not the absolute cosine value but the persistence of the condition ordering under a different representation model.

The embedding geometry also shifts. In zero-shot generation, grouping by trauma category dominates grouping by person-specific cognitive signature: the formulation-gap/trauma-gap ratio is 0.07. It rises to 0.48 under the full condition. The supplied structure appears to pull models away from generic trauma templates and toward the particular cognitive configuration assigned to the persona.

Clinical reviewers see related improvements. The largest reported expert contrast is for importance, rising from 2.23 to 2.94 with $d_z=1.98$, alongside gains in threat, appraisals, memory, and coping-strategy coverage. In a separate blinded study with 100 licensed clinicians, full-condition vignettes score more than one point above zero-shot on 1–3 scales for clarity, relevance, and importance. Perceived AI authorship falls from 77.8% for zero-shot cases to 14.7% for full cases.

These results connect structural control to reader-facing quality. They do not show that lower AI detection is itself desirable; here it functions as one measure of how natural and clinically coherent the cases appeared to practitioners.

Agreement is not the same as validation

One of the more reusable findings concerns evaluation design. The paper reports that LLM judges can agree with each other near ceiling while agreement between humans and LLMs on formulation content deteriorates sharply in zero-shot generation.

Consensus among model judges therefore cannot substitute for an external specification. Several evaluators may simply share the same blind spot.

Cognaptus inference: teams generating governed synthetic content should distinguish three layers of control. First, define the intended structure before generation. Second, test the output against that structure using an evaluator that is not merely the generator approving its own work. Third, use human review for properties that remain difficult to encode formally.

Forma partially implements this pattern, but its in-loop Crafter and Validator use the same model. Independent validation appears only in the research evaluation. A production implementation would have reason to separate those roles more aggressively.

What synthetic-data teams can borrow

For clinical education platforms, graph-backed cases could provide standardized stimuli where the mechanism being trained is known in advance. For healthcare-AI evaluation teams, the same idea creates benchmark items with an auditable gold structure rather than only a reference answer.

The transferable mechanism is not specifically a PTSD graph. It is the conversion of an intended latent structure into explicit generation obligations.

That also changes model-selection decisions. A cheaper generator may be unattractive if it regularly violates control-flow instructions, omits required relations, or increases the number of validation retries. Generation price alone does not capture the cost of reliable structured output.

The paper’s fairness analysis points in the same direction, though more cautiously. Under the tested single demographic axes, full formulation reduces disparities in perceived quality, with full-condition disparate-impact ratios above the four-fifths threshold and maximum pairwise Cohen’s $d$ no larger than 0.12. This supports controlled output-equivariance testing. It does not address intersectional effects, textual stereotyping, or broader representational harms.

Auditability is not clinical representativeness

Forma demonstrates that an LLM can populate a specified theory-grounded structure and leave enough evidence in the prose for that structure to be recovered.

It does not demonstrate that the sampled personas reproduce real PTSD populations. The graphs are programmatically generated, the content is English-only, and the system instantiates one cognitive theory whose assumptions and omissions are inherited by the generated cases. The clinician panel is also concentrated in the U.S., U.K., Canada, and Australia.

Most importantly, no patient outcomes, downstream decision-support performance, or population validity are tested. The authors do not establish these vignettes as suitable training data for patient-care clinical decision-support models.

The stronger conclusion is narrower and more actionable. High-stakes synthetic generation becomes easier to govern when teams specify the relationships that must survive, test for those relationships after generation, and keep synthetic provenance visible. Forma provides substantial evidence for that workflow inside a controlled synthetic PTSD setting.

Fluency can make generated content look finished. A recoverable specification makes it inspectable.

Cognaptus: Automate the Present, Incubate the Future.