Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection

View on arXiv ← Back to list

Authors: Liang Chen, Yong Zhang, Yibing Song, Lingqiao Liu, Jue Wang

Published: 2022-03-23 05:52:23+00:00

AI Summary

This paper tackles the challenge of generalizable deepfake detection by proposing a self-supervised learning approach. It leverages adversarial training to synthesize diverse augmented forgeries and a multi-task learning framework that encourages the model to predict forgery configurations, thereby improving sensitivity to various forgery types.

Abstract

Recent studies in deepfake detection have yielded promising results when the training and testing face forgeries are from the same dataset. However, the problem remains challenging when one tries to generalize the detector to forgeries created by unseen methods in the training dataset. This work addresses the generalizable deepfake detection from a simple principle: a generalizable representation should be sensitive to diverse types of forgeries. Following this principle, we propose to enrich the diversity of forgeries by synthesizing augmented forgeries with a pool of forgery configurations and strengthen the sensitivity to the forgeries by enforcing the model to predict the forgery configurations. To effectively explore the large forgery augmentation space, we further propose to use the adversarial training strategy to dynamically synthesize the most challenging forgeries to the current model. Through extensive experiments, we show that the proposed strategies are surprisingly effective (see Figure 1), and they could achieve superior performance than the current state-of-the-art methods. Code is available at url{https://github.com/liangchen527/SLADD}.

Key findings

The proposed method significantly outperforms state-of-the-art deepfake detection methods on multiple benchmark datasets, demonstrating superior generalizability. Ablation studies confirm the effectiveness of adversarial training and the self-supervised tasks in improving performance across different compression levels and forgery types.

Approach

The authors propose a framework that uses a synthesizer network to generate diverse augmented forgeries based on a pool of forgery configurations. A detector network is then trained to predict both whether an image is a deepfake and its associated forgery configurations. Adversarial training dynamically generates challenging forgeries for the detector.

Datasets

Faceforensics++ (FF++) for training, CelebDF, Deepfake Detection Challenge (DFDC), and DeeperForensics-1.0 (DF1.0) for testing.

Model(s)

Modified Xception network for both the synthesizer and detector. AM-Softmax loss is used for the main classification task, and L1 loss is used for other auxiliary tasks.

Author countries

Australia, China

← Previous