From Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakes

Authors: Long Ma, Zhiyuan Yan, Jin Xu, Yize Chen, Qinglang Guo, Zhen Bi, Yong Liao, Hui Lin

Published: 2025-04-07 08:34:28+00:00

AI Summary

This paper proposes a universal deepfake detection framework by identifying two generalizable artifacts: Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA). It introduces a novel data-level pseudo-fake creation framework, FIA-USA, which uses super-resolution and image-level self-blending to generate samples exhibiting these artifacts. Training a standard image classifier on this pseudo-fake data enables robust generalization to unseen deepfakes.

Abstract

Detecting deepfakes has been an increasingly important topic, especially given the rapid development of AI generation techniques. In this paper, we ask: How can we build a universal detection framework that is effective for most facial deepfakes? One significant challenge is the wide variety of deepfake generators available, resulting in varying forgery artifacts (e.g., lighting inconsistency, color mismatch, etc). But should we ``teach the detector to learn all these artifacts separately? It is impossible and impractical to elaborate on them all. So the core idea is to pinpoint the more common and general artifacts across different deepfakes. Accordingly, we categorize deepfake artifacts into two distinct yet complementary types: Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA). FIA arise from the challenge of generating all intricate details, inevitably causing inconsistencies between the complex facial features and relatively uniform surrounding areas. USA, on the other hand, are the inevitable traces left by the generator's decoder during the up-sampling process. This categorization stems from the observation that all existing deepfakes typically exhibit one or both of these artifacts. To achieve this, we propose a new data-level pseudo-fake creation framework that constructs fake samples with only the FIA and USA, without introducing extra less-general artifacts. Specifically, we employ a super-resolution to simulate the USA, while design a Blender module that uses image-level self-blending on diverse facial regions to create the FIA. We surprisingly found that, with this intuitive design, a standard image classifier trained only with our pseudo-fake data can non-trivially generalize well to unseen deepfakes.

Key findings

The proposed method significantly outperforms state-of-the-art detection techniques on both traditional and generative deepfake datasets, demonstrating superior generalization capabilities across over 58 distinct forgery methods. It also exhibits enhanced robustness to various unseen perturbations like Gaussian Blur and JPEG compression. Ablation studies confirm that all proposed components—FIA-USA, AFFS, and RCR—contribute positively to the overall detection performance.

Approach

The authors categorize deepfake artifacts into Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA). They propose FIA-USA, a data augmentation framework that creates pseudo-fakes by combining multi-type mask synthesis (for FIA) with multi-modal reconstruction using Autoencoder and Super-Resolution models (for USA). This framework is complemented by Automatic Forgery-aware Feature Selection (AFFS) and Region-aware Contrastive Regularization (RCR) to enhance a standard image classifier's generalization.

Datasets

FaceForensics++ (FF++), Deepfake Detection (DFD), Deepfake Detection Challenge (DFDC), preview version of DFDC (DFDCP), CelebDF (CDF), DeepfakeBench, Diffusion Facial Forgery (DiFF), DF40

Model(s)

EfficientNetB4, Autoencoder (AE), Super-Resolution (SR) models, Feature Pyramid Networks (FPN), ResNet

Author countries

China

← Previous