One Model, Two Label Spaces, and a Boundary Tradeoff

TL;DR for operators

A healthcare AI team may want one segmentation system for uterine ultrasound and MRI, even when the datasets identify different anatomical structures. Topology-informed Prompt-conditioned Universal Segmentation (TPUS) addresses this problem by combining dataset-specific input processing with shared feature learning and class-conditioned prediction.¹

Across 82 held-out cases from two datasets, TPUS leads seven competing complete methods on segmentation overlap and average boundary distance. Yet its improvements over the strongest baseline are modest.

The most revealing finding comes from an ablation: the complete model achieves the best MRI overlap score, while a simpler configuration produces a better average boundary distance.

For healthcare AI developers, TPUS offers a design for consolidating heterogeneous segmentation tasks. It does not establish that the most complex configuration is always preferable, that unfamiliar anatomical labels can be segmented reliably, or that the system is ready for clinical deployment.

One anatomy does not mean one segmentation task

A medical imaging team maintains two segmentation workflows. One identifies the uterine region in three-dimensional ultrasound. Another processes MRI scans, distinguishing the uterine wall, cavity, myoma, and Nabothian cyst.

Both concern uterine anatomy, but their images and annotation requirements differ. The ultrasound workflow produces a binary segmentation; the MRI workflow must distinguish several anatomical and pathological structures.

Combining these tasks under one model introduces two requirements: shared representations must accommodate different imaging appearances, while output predictions must preserve each dataset’s label definitions.

There is a separate measurement problem. A predicted region can overlap substantially with its reference annotation while placing parts of its boundary inaccurately. Dice measures spatial overlap; average symmetric surface distance (ASSD) measures average boundary separation. Higher Dice and lower ASSD are desirable, but neither metric guarantees improvement in the other.

That distinction matters when a model’s outputs may eventually support anatomical measurements or clinical image review.

Shared feature learning without forcing identical labels

TPUS separates the segmentation process into responsibilities that can be shared and responsibilities that remain dataset-dependent.

Each imaging dataset first passes through its own convolutional input stem. This transforms ultrasound or MRI inputs into a compatible feature representation before processing by a shared U-shaped encoder-decoder.

The shared network includes graph-based TopoBlocks adapted from Pool GNN and Swin GNN operations. These blocks model spatial relationships through pooled dynamic nearest-neighbor graphs and shifted three-dimensional windows, supplementing conventional convolutional feature extraction.

The output mechanism handles a different source of heterogeneity: label identity.

For each predefined class, including background, TPUS constructs a task-aware text prompt. A frozen CLIP text encoder converts the prompt into an embedding. The model then combines that class representation with features extracted from the current image.

A trainable controller generates 153 convolution parameters for each image-class combination. Three dynamically parameterized convolutional layers use these parameters to produce a class-specific prediction map. The resulting maps are combined according to the originating dataset’s native label vocabulary.

This arrangement allows the same feature-processing backbone to serve different segmentation tasks without requiring a single fixed set of foreground classes.

Importantly, TPUS is not a paired ultrasound–MRI fusion system. Each training iteration processes a volume from one dataset through its corresponding input stem. The common resource is the learned model, not simultaneous access to both imaging modalities for a patient.

Topology supervision is conditional on the annotation vocabulary

TPUS adds topology-interaction supervision to ordinary Dice and cross-entropy training losses.

The mechanism examines local three-dimensional neighborhoods for predicted relationships that violate foreground-class exclusion rules. Voxels identified as critical receive an additional classification penalty during training.

This supervision is applied at four decoder resolutions. It aims to discourage specified local structural inconsistencies, rather than proving that every segmented anatomical structure has globally correct topology.

Its applicability depends directly on the available labels.

The MRI dataset contains four foreground classes, producing six pairwise exclusion relations. The ultrasound dataset contains only one foreground class, so its exclusion set is empty and the topology-interaction penalty is zero.

Consequently, any improvement on the binary ultrasound task cannot be attributed directly to a nonzero topology-interaction penalty. Graph-based feature reasoning and topology-interaction supervision are distinct components with different roles.

For developers designing anatomical constraints, this is an important implementation detail: a topology-aware loss is only operationally meaningful when the annotation structure provides relationships for it to enforce.

The benchmark favors TPUS, but the margins deserve attention

The researchers evaluate TPUS on 404 cases: 138 ultrasound cases from UterUS and 266 MRI cases from the Uterine Myoma MRI Dataset (UMD). The held-out tests contain 28 and 54 cases, respectively.

Under shared preprocessing and evaluation conditions, TPUS leads seven comparison methods, including MultiTalent, the strongest complete baseline.

Source: Section 3.2, Table 1. Higher Dice and lower ASSD are better.

On ultrasound, the Dice improvement is 0.003 and the ASSD reduction is 0.016 mm. On MRI, the corresponding improvements are 0.013 and 0.091 mm.

The paper reports statistically significant ASSD comparisons against the evaluated methods using paired Wilcoxon tests. Equivalent statistical support is not established for every Dice improvement.

The reported UMD ASSD standard deviation for TPUS is 4.680 mm, substantially larger than its mean of 1.684 mm. That dispersion warrants examination of case-level boundary errors rather than reliance on the mean alone.

These are favorable predictive benchmarking results. Their magnitude does not establish a corresponding clinical benefit.

The ablation exposes a boundary-accuracy tradeoff

The UMD ablation isolates architectural combinations to investigate their contributions. It also reveals why selecting the complete system solely because it has the highest Dice would be premature.

Source: Section 3.3, Table 2; all 54 UMD test cases. Base is the retrained MultiTalent configuration.

The complete architecture improves overlap relative to every tested configuration. However, adding topology-interaction supervision alone to the Base produces a lower average boundary distance than the complete model.

The difference is not proof that the simpler configuration is universally superior. The ablation does not isolate every possible component interaction or establish statistical significance for each configuration pair.

It does demonstrate that improvements are metric-dependent.

A separate prompt-encoding comparison on UterUS serves a narrower purpose. CLIP-based conditioning achieves Dice 0.898 and ASSD 0.432 mm, versus 0.896 and 0.448 mm with random embeddings. This favors pretrained text conditioning under the tested setting, but does not establish reliable segmentation of previously unseen classes from arbitrary descriptions.

What a healthcare AI team can adopt—and what remains unproven

Cognaptus inference: The most transferable contribution is the separation of dataset-specific input adaptation, shared feature learning, and label-conditioned output prediction.

For teams managing several imaging datasets, that architecture offers a plausible alternative to maintaining fully independent models. Whether it actually reduces training, maintenance, or infrastructure costs remains unmeasured.

Component selection should follow the intended application. A team prioritizing overall anatomical overlap may prefer a configuration different from one whose downstream measurements are particularly sensitive to boundary placement. Both Dice and ASSD should inform evaluation, alongside case-level failure analysis.

The validation boundary is substantial. TPUS is tested on one ultrasound dataset and one MRI dataset, with different segmentation targets. UterUS excludes cases with several acquired abnormalities, while UMD includes patients with confirmed fibroids. The study does not demonstrate performance on unseen institutions, scanners, acquisition protocols, or label vocabularies.

There is also a reproducibility qualification. The public TPUS implementation provides code and synthetic testing utilities, but the source package reports that original medical data, case-level splits, checkpoints, and a completed full-training reproduction are not distributed.

Before deployment, developers would need independent validation on the intended clinical population, assessment of failure modes and boundary accuracy, and evidence that segmentation performance supports the specific downstream clinical decisions. None of those outcomes is established here.

A common architecture still requires task-specific standards

TPUS shows that heterogeneous imaging tasks can share substantial model infrastructure while retaining their own input and output semantics. Its benchmark results support that design within the two evaluated datasets.

The component experiments add a necessary qualification: the highest-overlap model is not automatically the lowest-boundary-error model.

For healthcare AI platforms, the relevant engineering question is therefore not merely how many segmentation tasks one architecture can accommodate. It is whether the shared system preserves the accuracy requirements of each task—and whether the additional architectural complexity improves the metrics that matter to its eventual use.

Cognaptus: Automate the Present, Incubate the Future.