DiffMark: Diffusion-based Robust Watermark Against Deepfakes

View on arXiv ← Back to list

Authors: Chen Sun, Haiyang Sun, Zhiqing Guo, Yunfeng Diao, Liejun Wang, Dan Ma, Gaobo Yang, Keqin Li

Published: 2025-07-02 07:29:33+00:00

AI Summary

DiffMark is a novel robust watermarking framework based on diffusion models that embeds watermarks into facial images, making them resistant to Deepfake manipulations. It uses facial images and watermarks as conditions to guide the diffusion model's denoising process, improving robustness by simulating Deepfake attacks during training and adversarially guiding the sampling process.

Abstract

Deepfakes pose significant security and privacy threats through malicious facial manipulations. While robust watermarking can aid in authenticity verification and source tracking, existing methods often lack the sufficient robustness against Deepfake manipulations. Diffusion models have demonstrated remarkable performance in image generation, enabling the seamless fusion of watermark with image during generation. In this study, we propose a novel robust watermarking framework based on diffusion model, called DiffMark. By modifying the training and sampling scheme, we take the facial image and watermark as conditions to guide the diffusion model to progressively denoise and generate corresponding watermarked image. In the construction of facial condition, we weight the facial image by a timestep-dependent factor that gradually reduces the guidance intensity with the decrease of noise, thus better adapting to the sampling process of diffusion model. To achieve the fusion of watermark condition, we introduce a cross information fusion (CIF) module that leverages a learnable embedding table to adaptively extract watermark features and integrates them with image features via cross-attention. To enhance the robustness of the watermark against Deepfake manipulations, we integrate a frozen autoencoder during training phase to simulate Deepfake manipulations. Additionally, we introduce Deepfake-resistant guidance that employs specific Deepfake model to adversarially guide the diffusion sampling process to generate more robust watermarked images. Experimental results demonstrate the effectiveness of the proposed DiffMark on typical Deepfakes. Our code will be available at https://github.com/vpsg-research/DiffMark.

Key findings

DiffMark outperforms existing methods in terms of both watermark invisibility and robustness against Deepfake manipulations and common distortions. The Deepfake-resistant guidance further improves robustness, particularly against SimSwap. Results are consistent across CelebA-HQ and LFW datasets, demonstrating generalization.

Approach

DiffMark uses a diffusion model to generate watermarked images by taking the facial image and watermark as conditions to guide the denoising process. It enhances robustness by incorporating a frozen autoencoder during training and using Deepfake-resistant guidance during sampling.

Datasets

CelebA-HQ, LFW

Model(s)

Diffusion model (U-Net as backbone), VQGAN (frozen autoencoder), DDIM sampler

Author countries

China, USA

← Previous