TriniMark: A Robust Generative Speech Watermarking Method for Trinity-Level Attribution

View on arXiv ← Back to list

Authors: Yue Li, Weizhi Liu, Dongdong Lin

Published: 2025-04-29 08:23:28+00:00

AI Summary

This paper introduces TriniMark, a robust generative speech watermarking method for authenticating synthetic speech and tracing it back to the diffusion model and user. It achieves this through a two-stage process: pre-training a lightweight encoder-decoder and then fine-tuning the diffusion model using a waveform-guided strategy.

Abstract

The emergence of diffusion models has facilitated the generation of speech with reinforced fidelity and naturalness. While deepfake detection technologies have manifested the ability to identify AI-generated content, their efficacy decreases as generative models become increasingly sophisticated. Furthermore, current research in the field has not adequately addressed the necessity for robust watermarking to safeguard the intellectual property rights associated with synthetic speech and generative models. To remedy this deficiency, we propose a textbf{ro}bust generative textbf{s}peech wattextbf{e}rmarking method (TriniMark) for authenticating the generated content and safeguarding the copyrights by enabling the traceability of the diffusion model. We first design a structure-lightweight watermark encoder that embeds watermarks into the time-domain features of speech and reconstructs the waveform directly. A temporal-aware gated convolutional network is meticulously designed in the watermark decoder for bit-wise watermark recovery. Subsequently, the waveform-guided fine-tuning strategy is proposed for fine-tuning the diffusion model, which leverages the transferability of watermarks and enables the diffusion model to incorporate watermark knowledge effectively. When an attacker trains a surrogate model using the outputs of the target model, the embedded watermark can still be learned by the surrogate model and correctly extracted. Comparative experiments with state-of-the-art methods demonstrate the superior robustness of our method, particularly in countering compound attacks.

Key findings

TriniMark demonstrates superior robustness compared to state-of-the-art methods against various individual and compound attacks, maintaining high watermark extraction accuracy even at a high capacity (500 bps). The waveform-guided fine-tuning strategy improves both watermark robustness and the quality of generated watermarked speech.

Approach

TriniMark embeds watermarks into the time-domain features of speech using a lightweight encoder and a temporal-aware gated convolutional network decoder. A waveform-guided fine-tuning strategy is used to integrate watermark knowledge into the diffusion model during training, ensuring robustness against attacks.

Datasets

LJSpeech, LibriTTS, LibriSpeech

Model(s)

DiffWave, PriorGrad

Author countries

China

← Previous