TL;DR for operators

A synthetic-speech watermark is only operationally valuable if its identifier remains recoverable after distribution, compression, and editing. Generation-Aware Mel-Domain Watermarking (GAMW) places identifiers inside the speech-generation pipeline rather than modifying the completed recording. Across two TTS systems, it achieves approximately 91% average bitwise recovery under eight multi-stage audio-processing pipelines, outperforming four evaluated post-generation methods.

The architecture also extracts identifiers without the original recording and leaves the pretrained speech generator frozen. However, the encoder and extractor require backbone-specific training, and accuracy decreases as payload length increases.

For TTS providers and content-governance teams, the practical opportunity is stronger identifier recovery through processing pipelines. Production attribution still requires separate tests for complete-message recovery, authentication, false acceptance, perceptual quality, and deliberate watermark manipulation.

The recording arrives, but does its identifier?

A text-to-speech service can embed an identifier when it generates a recording. That recording may subsequently pass through compression, format conversion, and other audio processing before reaching a platform responsible for identifying its source.

A watermark that works perfectly on the original recording may become difficult to recover after these transformations. Testing each operation separately does not necessarily reveal what happens when several are applied in sequence.

The operational question is therefore whether the identifier survives the processing actually encountered between generation and verification.

Yihang Zhang and colleagues investigate this problem in GAMW: Generation-Aware Mel-Domain Watermarking for TTS Traceability.1 Their results show that moving watermark insertion into the speech-generation process can substantially improve recoverability under the evaluated processing conditions.

The largest comparative advantage appears when transformations are combined.

Evaluation GAMW: CosyVoice2 GAMW: GLM-TTS Strongest baseline: TimbreWM
15 individual perturbations 98.54% 98.20% 89.25% / 89.00%
8 compound pipelines 91.45% 90.71% 67.74% / 67.40%

Source: Section 4.4; Appendix C, Table 8; Appendix D, Table 9. Values are average bitwise decoding accuracy, equally weighted across tested conditions. TimbreWM results are ordered CosyVoice2 / GLM-TTS.

GAMW’s advantage over TimbreWM increases from approximately 9.2 percentage points under individual perturbations to 23.3–23.7 points under compound processing. It leads all four evaluated baselines in every compound condition on both backbones.

Yet the most difficult tested pipeline—EnCodec followed by Opus and quantization—reduces GAMW’s bitwise accuracy to 83.32% and 82.18%.

Comparative superiority therefore coexists with substantial residual decoding errors. The baselines also carry different native payload lengths, making the comparison informative but not strictly capacity-matched.

Place the identifier before speech becomes a waveform

Many TTS systems first produce a mel-spectrogram: a time-frequency representation describing the sound to be synthesized. A vocoder then converts that representation into an audible waveform.

Conventional post-generation watermarking modifies the waveform after synthesis. GAMW instead inserts its watermark between the acoustic model and the vocoder.

Its encoder constructs a content-dependent residual using cross-attention. Mel frames act as queries, while learned watermark-bit representations supply keys and values. A learned time-frequency mask distributes the residual across the acoustic representation, and a global coefficient controls its overall strength.

The modified mel-spectrogram passes through a frozen vocoder, producing speech with the identifier incorporated during waveform synthesis.

This arrangement matters for two reasons.

First, watermark placement can adapt to acoustic content rather than applying an independently constructed signal to completed audio. Second, the pretrained acoustic model and vocoder need not be updated while the watermark components are trained.

Verification works in the opposite direction. An extractor processes the received audio, constructs acoustic features, and predicts the embedded bits through convolutional and Transformer components.

It does not require the original waveform or clean mel-spectrogram. This is reference-free extraction—a practical property when a platform receives distributed audio but cannot access the source recording.

A separate embedding-position ablation favors the evaluated mel-domain implementation: 97.95% average individual-perturbation accuracy versus 90.48% for its waveform-domain alternative. The mel implementation also achieves better PESQ, ViSQOL, and predicted speech-quality scores.

This is supporting architectural evidence, not proof that insertion position alone caused the improvement. The two variants use different representation-specific components and fidelity losses.

The decoder must learn its weakest bits

Placing information before the vocoder does not eliminate uneven decoding difficulty.

Some watermark bits may become reliably recoverable early in training while others remain difficult. Optimizing ordinary average binary cross-entropy can continue spending substantial effort on predictions that are already correct.

GAMW combines two adjustments.

Focal modulation reduces the influence of easy, high-confidence sample-bit predictions. Dynamic per-bit weighting assigns more weight to bit positions with lower accuracy in the current mini-batch.

The complete objective also penalizes differences between the original and modified mel-spectrograms, balancing recoverability against acoustic fidelity.

A controlled two-by-two loss ablation examines these mechanisms during the first clean training epoch.

The combined focal-plus-per-bit-weighting objective reaches both specified validation thresholds—overall accuracy of at least 99% and minimum per-bit accuracy of at least 95%—at step 700. Variants omitting focal modulation or per-bit weighting reach the criterion at steps 1,500 and 1,700, respectively. Ordinary binary cross-entropy does not reach it within that epoch.

This supports the contribution of both components to early optimization efficiency. It does not independently show that each component improves long-run resistance to compound perturbations.

For engineering teams, the relevant measurement is not only mean recovery across all bits. The weakest bit positions can determine whether an identifier remains usable.

The paper’s failure analysis reinforces this point. Under EnCodec followed by MP3, Bit 4 falls to approximately 66% accuracy, while another codec-and-quantization pipeline disproportionately affects Bits 13 and 15.

Different processing chains produce different failure patterns.

More identifier bits introduce another reliability trade-off

GAMW’s primary robustness experiments use 16-bit payloads. Under clean conditions, the system recovers all bits correctly on both evaluated TTS backbones.

At 100 bits, clean-condition accuracy remains high: 99.78% on CosyVoice2 and 99.03% on GLM-TTS. However, PESQ and ViSQOL decline relative to the 16-bit configuration.

The robustness cost becomes visible earlier.

Increasing the payload from 16 to 32 bits reduces average individual-perturbation accuracy from 98.54% to 96.17% on CosyVoice2 and from 98.20% to 95.83% on GLM-TTS.

The 100-bit experiment establishes clean-condition capacity, not equivalent robustness under compression or sequential transformations. Compound-perturbation performance is reported for the primary 16-bit configuration.

Audio quality requires similar precision. GAMW achieves the highest ViSQOL and UTMOS-predicted mean opinion scores among the compared methods on both backbones, but it does not lead every quality metric. The reported MOS values are model predictions, not measurements from a human listening study.

For operators selecting an identifier format, payload length cannot be treated independently of acoustic quality and expected downstream processing.

Recoverable bits are not authenticated provenance

The central measurement in these experiments is bitwise decoding accuracy: the proportion of individual watermark bits predicted correctly.

It is not the percentage of complete identifiers recovered without error.

A 16-bit identifier containing one incorrect bit is not an exact match, even though its bitwise accuracy is 93.75%. The reported averages do not establish complete-message recovery probabilities or verification false-acceptance rates.

Nor does reference-free decoding establish that a recovered identifier genuinely originated from an authorized generator.

The experiments use predefined transformations and two separately trained, mel-based TTS backbones. They do not test adaptive removal, forgery, overwriting, or attacks targeting the extractor. Cross-backbone transfer is also untested. Evaluation includes only one neural codec family, EnCodec, and the total independent test-utterance count is not clearly reported.

These boundaries materially affect deployment decisions.

For a TTS provider operating a compatible mel-to-vocoder architecture, GAMW provides a concrete design candidate. An implementation evaluation should measure exact-message recovery across the provider’s actual codec sequences and payload requirements, alongside perceptual quality.

For a content platform using decoded identifiers to attribute synthetic recordings, additional controls are necessary: authenticated mapping to generation records, false-acceptance testing, handling of decoding failures, and adversarial-security evaluation.

These are Cognaptus operational implications drawn from the engineering results, not deployment outcomes demonstrated by the paper.

The next test belongs to the distribution pipeline

GAMW’s contribution is architectural as well as empirical. It demonstrates that embedding an identifier before vocoding, with content-adaptive placement and difficulty-aware training, can improve recovery after substantial audio processing.

Its strongest relative results occur under the sequential transformations that complicate downstream verification. Its remaining errors also show why the benchmark’s average bitwise accuracy cannot serve as a provenance guarantee.

For synthetic-speech operators, the next decision is to test generation-integrated watermarking against the complete route an audio recording takes—from synthesis through distribution to verification—with exact-message reliability and authenticated attribution evaluated separately.

Cognaptus: Automate the Present, Incubate the Future.


  1. Yihang Zhang and Weicai Yan and Qifei Zhang and Yangyang Wu and Tao Jin (2026). GAMW: Generation-Aware Mel-Domain Watermarking for TTS Traceability. arXiv:2610.15082. https://arxiv.org/abs/2610.15082 ↩︎