Heartbeat in Stereo: Why ECG AI Needs Both Contrast and Context

ECG models have a deceptively simple job: read a heartbeat and infer what might be wrong.

The real problem is that a heartbeat is not a single line of data. A standard 12-lead ECG is a coordinated view of cardiac electrical activity from multiple spatial angles. Meanwhile, the associated clinical report is not a clean label. It is a human-written summary: useful, compressed, inconsistent, and occasionally full of stylistic residue. Medicine, regrettably, still contains humans.

That is the setting for CG-DMER: Hybrid Contrastive-Generative Framework for Disentangled Multimodal ECG Representation Learning, an arXiv paper accepted by ICASSP 2026.¹ The paper’s central move is not simply “combine ECG and text.” That would be the obvious version, and the obvious version is precisely where multimodal medical AI often becomes messy.

CG-DMER argues for a more careful design: first preserve the structure inside the ECG signal, then learn useful semantics from the report, then align only the part of each modality that should actually be shared.

That order matters.

The wrong mental model: “just align ECGs with reports”

A tempting business interpretation of ECG-text learning is that clinical reports can act as cheap supervision. Pair each ECG with its report, train a model to align the waveform with the language, and suddenly the model understands cardiology. This is the healthcare version of “just add text and stir.”

The paper pushes against that intuition in two ways.

First, many ECG representation models are lead-agnostic. They may process the 12-lead signal in a way that weakens the distinction between different spatial views of the heart. That is a strange shortcut. In clinical interpretation, lead placement and cross-lead relationships are not decorative metadata; they are part of the diagnostic signal.

Second, many multimodal methods directly align ECG signals with free-text reports. Reports are valuable because they contain diagnostic context. They are also noisy because they are written in natural language, shaped by institutional habit, physician style, and what happened to be clinically salient at the time. Direct alignment can therefore pull the ECG representation toward text-specific artifacts rather than toward stable cardiac semantics.

CG-DMER’s answer is a three-part mechanism:

The important word is not “multimodal.” It is disentangled.

CG-DMER fixes the ECG representation before asking text to help

The first contribution is easy to underestimate because it happens before the glamorous ECG-text alignment stage.

CG-DMER tokenizes each ECG lead independently. It then adds lead-specific spatial embeddings and shared temporal embeddings. Instead of treating a time segment as one generic token, the model preserves which lead the token came from and where it sits in the time sequence.

This matters because ECG information is not only temporal. A rhythm abnormality may appear across time; a morphology pattern may become clinically meaningful because of how it appears across leads. If the model compresses away lead identity too early, it may still learn something useful, but it is being asked to rediscover spatial structure after the architecture has already blurred it. That is not impossible. It is merely inefficient, like asking an intern to reconstruct the org chart after deleting everyone’s job title.

The masked reconstruction task then forces the model to infer missing ECG patches from surrounding temporal and spatial context. In effect, the model learns what should plausibly appear in a given lead at a given time, conditioned on the rest of the ECG.

This is not just a pretraining trick. It is the mechanism that makes the later multimodal alignment less fragile. If the ECG encoder already understands lead-level structure, the text does not need to compensate for a weak signal representation. The report can add semantic guidance rather than serving as a crutch.

The report is useful, but not innocent

The second mechanism is masked report reconstruction. The paper follows the broader masked language modeling idea: hide parts of the report and train the model to recover them.

This does two things. It encourages the text branch to learn medical semantics, and it creates a generative objective that complements the ECG reconstruction task. The report branch is not merely a label lookup table. It becomes a representation learner in its own right.

But CG-DMER does not pretend that clinical text is pure truth. That is the more mature part of the design.

A report can say “normal sinus rhythm,” “possible left ventricular hypertrophy,” or “borderline ECG.” It can also omit details, use shorthand, reflect local reporting conventions, or combine observation with diagnostic caution. Directly collapsing this text into the ECG representation risks importing language noise into the signal model.

So the paper’s architecture separates modality-specific and modality-shared representations. The ECG keeps information that belongs to ECG morphology. The text keeps information that belongs to language and reporting style. Only the shared semantic components are aligned.

This is the practical lesson beyond cardiology: multimodal AI should not always merge modalities as early as possible. Sometimes the first job is to protect each modality from the other.

Disentanglement is the business-relevant move

The most interesting part of CG-DMER is not that it combines contrastive and generative learning. That is useful, but expected. The sharper contribution is that it uses disentanglement to decide what should be aligned.

The paper applies modality-shared and modality-specific encoders to both ECG and text representations. It also uses an orthogonality constraint to reduce redundancy between the shared and specific components. Then cross-modal contrastive learning is applied to the shared ECG-text representations.

In business terms, this is a design pattern for noisy enterprise data.

Healthcare vendors often work with structured signals and unstructured notes. Outside healthcare, the same pattern appears in machine logs and incident tickets, transaction records and analyst commentary, sensor streams and maintenance reports. The structured modality often carries precise operational evidence. The text modality carries interpretation, context, and human judgment. Blindly aligning them can create a representation that is neither clean signal nor clean language. A beautiful swamp, perhaps, but still a swamp.

CG-DMER’s principle is cleaner:

Preserve what is modality-specific. Align what is semantically shared. Do not let the report bully the waveform.

That principle is more important than the individual architecture choices, because it travels well.

What the experiments actually show

The paper evaluates CG-DMER on two downstream settings: linear probing and zero-shot classification. Both use macro AUC as the metric. The model is pretrained on MIMIC-IV-ECG, using 779,891 processed ECG-report pairs after preprocessing. Downstream evaluation covers PTB-XL, CPSC2018, and CSN, spanning more than 100 cardiac conditions.

The experimental evidence should be read in layers.

This separation matters. A paper can have strong benchmark evidence without becoming a product validation study. Confusing those two is how AI strategy decks become expensive fiction.

Linear probing: the label-efficiency story is the strongest result

Linear probing evaluates the quality of pretrained ECG representations by freezing the encoder and training a simple classifier on top. If performance is strong under this setup, the representation itself is doing useful work.

CG-DMER performs strongly across the reported datasets and label ratios. The paper emphasizes that with only 10% labeled data, CG-DMER surpasses all ECG-only self-supervised baselines trained with 100% labeled data. That is the business-relevant signal: better pretraining can reduce downstream annotation pressure.

Selected 100% label linear probing results show the pattern clearly:

The gains are not uniform in size. Some are modest; others are more substantial. That is normal. Representation learning improvements usually do not arrive as cinematic explosions. They arrive as a series of small, stubborn advantages across tasks.

For healthcare AI companies, the practical message is not “this model is ready for hospitals.” The more defensible message is: pretraining design can shift the economics of labeled-data dependence. If an ECG model can perform well with fewer downstream labels, the cost and time required to adapt it to a new diagnostic taxonomy may fall.

That is not a small thing. Expert annotation is one of the quiet tax collectors of medical AI.

Zero-shot classification: useful, but read the magnitude carefully

CG-DMER also reports zero-shot ECG classification results. Here, the model uses text prompts to classify ECGs without task-specific labeled examples. The paper uses GPT-4o to generate richer category descriptions rather than relying only on class names.

The average zero-shot AUC is 76.00, compared with 75.25 for MERL and 75.16 for C-MELT.

The improvement over the strongest baseline is +0.75 AUC points. That is not a revolution. It is a measured improvement in a difficult setting.

The more important interpretation is qualitative: CG-DMER’s disentangled alignment does not merely help linear probing; it also appears to help semantic transfer under zero-shot prompts. In other words, the shared representation is not just numerically better for supervised classifiers. It is also somewhat better at connecting ECG patterns with clinical language.

That matters for future product design, especially in low-label environments. But it should not be oversold. Zero-shot ECG classification is not the same as zero-risk clinical automation. The result supports representation quality, not unsupervised deployment in a cardiology workflow.

A boring distinction, yes. Also the kind that prevents lawsuits.

The ablation table explains why the mechanism matters

The ablation study is where the paper’s mechanism-first story becomes credible. Starting from a contrastive-only multimodal baseline, the authors add components step by step and report the effect on average 1% linear probing AUC and average zero-shot AUC.

Using the table values:

The largest early jump comes from spatial-temporal ECG masking. That supports the paper’s claim that ECG structure should be modeled carefully before multimodal alignment does its work.

The disentanglement step is also important. It delivers a meaningful improvement after both ECG and text reconstruction have already been added. That is the evidence behind the article’s main argument: multimodal learning is not only about adding more sources of information. It is about controlling how those sources contaminate or clarify each other.

Masked text reconstruction helps, but its incremental gain is smaller. That is not a weakness. It simply suggests that clinical text is most valuable when used as part of a disciplined multimodal system, not as a magic annotation replacement.

The sensitivity tests are engineering guidance, not a second thesis

The paper also tests patch number and masking ratio.

The optimal patch number is reported as 100. Larger patch numbers, such as 200, degrade performance more sharply than smaller values such as 25. The paper’s explanation is that excessive segmentation may introduce ambiguity by aggregating or fragmenting temporal information in less useful ways.

The masking ratio performs best at 75%, a moderately high level. That makes sense for masked modeling. If the task is too easy, the model can reconstruct missing parts using shallow local cues. If it is too hard, the reconstruction signal may become unstable. A 75% mask ratio appears to create enough difficulty to force useful representation learning.

These tests should be treated as sensitivity analysis. They do not prove that 100 patches and 75% masking are universal constants handed down from the ECG gods. They show that the method’s performance depends on reconstruction granularity and task difficulty, and that the chosen configuration is empirically reasonable in this setup.

The t-SNE visualization plays a different role. It shows CG-DMER producing more compact and separated clusters for selected CSN diagnostic categories compared with SimCLR, ST-MEM, and MERL. This is useful as an interpretive illustration. It is not the core evidence. Pretty clusters are nice. Benchmarks still pay the rent.

What this means for healthcare AI vendors

The business value of CG-DMER is not “better ECG classification” in the abstract. The clearer value is cheaper adaptation.

A vendor building ECG models faces several recurring costs:

1. Acquiring labeled ECG datasets for each target condition or workflow;
2. Mapping local clinical terminology into model-compatible classes;
3. Handling variation across hospitals, devices, and reporting styles;
4. Maintaining performance as the product expands across geographies and clinical settings.

A stronger pretrained representation can reduce the pressure on the first two. If the model already encodes useful ECG morphology and clinical semantics, then downstream classifiers may need fewer labels to reach acceptable performance. Zero-shot or low-shot workflows may also become more plausible for triage, exploration, or internal decision support.

But this is an inference from benchmark behavior, not something the paper directly validates in hospital operations.

A practical adoption pathway would look like this:

The model is best understood as a representation layer, not as a finished clinical product. That distinction is commercially important. A representation layer can support many downstream products. A single classifier is narrower and easier to obsolete.

Boundaries that affect practical interpretation

There are several limits to keep in view.

First, the results are benchmark results. PTB-XL, CPSC2018, CSN, and MIMIC-IV-ECG are valuable public resources, but public benchmark success is not the same as prospective clinical validation.

Second, zero-shot evaluation depends on prompt design. The paper uses GPT-4o to generate richer clinical category descriptions. That is reasonable, but it means zero-shot performance is partly a function of the prompt-generation pipeline. In production, prompt governance becomes part of model governance. Annoying, but true.

Third, clinical reports are not evenly reliable across institutions. A model pretrained on paired ECG-report data may inherit reporting conventions from the source dataset. Disentanglement helps reduce modality bias, but it does not abolish it.

Fourth, the paper focuses on representation quality rather than interpretability for clinical decision-making. Better representations can improve downstream performance, but clinicians and regulators may still ask why a model made a specific recommendation. “The embedding looked healthy” is not a satisfying answer, even when said confidently.

These limitations do not weaken the paper’s contribution. They locate it. CG-DMER is a serious representation learning paper, not a clinical deployment certificate.

The broader lesson: multimodal AI needs boundaries before bridges

CG-DMER is useful because it resists a lazy version of multimodal learning.

It does not say: ECG plus text equals intelligence.

It says: ECG has spatial-temporal structure. Reports have clinical semantics and language noise. Learn each properly. Separate what belongs to each modality. Align only the shared meaning.

That is a more general enterprise AI principle. The future of multimodal systems will not be won by dumping every available data source into one embedding space and hoping scale performs spiritual cleansing. Some data should be aligned. Some should be preserved. Some should be filtered. Some should be ignored. The architecture has to know the difference.

For ECG AI, that means the heartbeat needs both contrast and context. Contrast sharpens discrimination. Context supplies clinical meaning. Disentanglement keeps the two from becoming a very expensive blur.

And in medical AI, avoiding expensive blur is already progress.

Cognaptus: Automate the Present, Incubate the Future.