Facial Features Matter: a Dynamic Watermark based Proactive Deepfake Detection Approach

Authors: Shulin Lan, Kanlin Liu, Yazhou Zhao, Chen Yang, Yingchao Wang, Xingshan Yao, Liehuang Zhu

Published: 2024-11-22 08:49:08+00:00

AI Summary

This paper introduces FaceProtect, a proactive deepfake detection method that utilizes dynamic watermarks based on facial features to overcome generalization and security issues in existing approaches. It proposes a GAN-based One-way Dynamic Watermark Generating Mechanism (GODWGM) for creating irreversible, dynamic watermarks from 128-dimensional facial feature vectors, along with a Watermark-based Verification Strategy (WVS) for secure embedding and recovery. Experimental results demonstrate the method's exceptional detection performance and practicality on images altered by various deepfake techniques.

Abstract

Current passive deepfake face-swapping detection methods encounter significance bottlenecks in model generalization capabilities. Meanwhile, proactive detection methods often use fixed watermarks which lack a close relationship with the content they protect and are vulnerable to security risks. Dynamic watermarks based on facial features offer a promising solution, as these features provide unique identifiers. Therefore, this paper proposes a Facial Feature-based Proactive deepfake detection method (FaceProtect), which utilizes changes in facial characteristics during deepfake manipulation as a novel detection mechanism. We introduce a GAN-based One-way Dynamic Watermark Generating Mechanism (GODWGM) that uses 128-dimensional facial feature vectors as inputs. This method creates irreversible mappings from facial features to watermarks, enhancing protection against various reverse inference attacks. Additionally, we propose a Watermark-based Verification Strategy (WVS) that combines steganography with GODWGM, allowing simultaneous transmission of the benchmark watermark representing facial features within the image. Experimental results demonstrate that our proposed method maintains exceptional detection performance and exhibits high practicality on images altered by various deepfake techniques.

Key findings

The proposed FaceProtect method achieved exceptional deepfake detection performance and generalization across various deepfake techniques, significantly outperforming several passive detection baselines. It effectively demonstrated that dynamic facial feature-based watermarks are robust for detection, imperceptible to the human eye, yet show notable deviations after deepfake manipulation. The use of grayscale images for watermarking also proved superior to binary sequences in terms of recovery quality and robustness.

Approach

The method proactively detects deepfakes by embedding dynamic watermarks, derived from 128-dimensional facial feature vectors via a GAN-based one-way mechanism (GODWGM), into original images using steganography. During detection, it extracts the hidden watermark and compares its cosine similarity with a new watermark generated from the image's current facial features to identify alterations. This approach leverages the inherent changes in facial features caused by deepfake manipulation.

Datasets

CelebA, MNIST, InfoSwap (for fake generation), SimSwap (for fake generation), StyleGAN2 (for fake generation), AttGAN (for fake generation)

Model(s)

WGAN GP (for GODWGM), Dlib (for facial feature extraction), U-Net and SENet (for WVS hiding network), 6-layer convolutional network (for WVS recovery network)

Author countries

China

← Previous