Fake It till You Make It: Curricular Dynamic Forgery Augmentations towards General Deepfake Detection

Authors: Yuzhen Lin, Wentang Song, Bin Li, Yuezun Li, Jiangqun Ni, Han Chen, Qiushi Li

Published: 2024-09-22 13:51:22+00:00

AI Summary

This paper introduces CDFA, a novel deepfake detection method that jointly trains a detector with a forgery augmentation policy network. CDFA progressively applies forgery augmentations following a monotonic curriculum and dynamically selects augmentations for optimal generalization, significantly improving cross-dataset and cross-manipulation performance.

Abstract

Previous studies in deepfake detection have shown promising results when testing face forgeries from the same dataset as the training. However, the problem remains challenging when one tries to generalize the detector to forgeries from unseen datasets and created by unseen methods. In this work, we present a novel general deepfake detection method, called textbf{C}urricular textbf{D}ynamic textbf{F}orgery textbf{A}ugmentation (CDFA), which jointly trains a deepfake detector with a forgery augmentation policy network. Unlike the previous works, we propose to progressively apply forgery augmentations following a monotonic curriculum during the training. We further propose a dynamic forgery searching strategy to select one suitable forgery augmentation operation for each image varying between training stages, producing a forgery augmentation policy optimized for better generalization. In addition, we propose a novel forgery augmentation named self-shifted blending image to simply imitate the temporal inconsistency of deepfake generation. Comprehensive experiments show that CDFA can significantly improve both cross-datasets and cross-manipulations performances of various naive deepfake detectors in a plug-and-play way, and make them attain superior performances over the existing methods in several benchmark datasets.