Suppressing Gradient Conflict for Generalizable Deepfake Detection

Authors: Ming-Hui Liu, Harry Cheng, Xin Luo, Xin-Shun Xu

Published: 2025-07-29 06:48:22+00:00

Comment: V1

AI Summary

This paper addresses the problem of degraded deepfake detection performance when models are trained on both original and online synthesized fake images, a phenomenon attributed to gradient conflicts during backpropagation. The proposed Conflict-Suppressed Deepfake Detection (CS-DFD) framework mitigates this issue through two modules: Update Vector Search (UVS), which finds a conflict-free update vector, and Conflict Gradient Reduction (CGR), which uses a novel Conflict Descent Loss to enforce a low-conflict feature embedding space. This synergy alleviates gradient interference, leading to state-of-the-art performance in both in-domain accuracy and cross-domain generalization.

Abstract

Robust deepfake detection models must be capable of generalizing to ever-evolving manipulation techniques beyond training data. A promising strategy is to augment the training data with online synthesized fake images containing broadly generalizable artifacts. However, in the context of deepfake detection, it is surprising that jointly training on both original and online synthesized forgeries may result in degraded performance. This contradicts the common belief that incorporating more source-domain data should enhance detection accuracy. Through empirical analysis, we trace this degradation to gradient conflicts during backpropagation which force a trade-off between source domain accuracy and target domain generalization. To overcome this issue, we propose a Conflict-Suppressed Deepfake Detection (CS-DFD) framework that explicitly mitigates the gradient conflict via two synergistic modules. First, an Update Vector Search (UVS) module searches for an alternative update vector near the initial gradient vector to reconcile the disparities of the original and online synthesized forgeries. By further transforming the search process into an extremum optimization problem, UVS yields the uniquely update vector, which maximizes the simultaneous loss reductions for each data type. Second, a Conflict Gradient Reduction (CGR) module enforces a low-conflict feature embedding space through a novel Conflict Descent Loss. This loss penalizes misaligned gradient directions and guides the learning of representations with aligned, non-conflicting gradients. The synergy of UVS and CGR alleviates gradient interference in both parameter optimization and representation learning. Experiments on multiple deepfake benchmarks demonstrate that CS-DFD achieves state-of-the-art performance in both in-domain detection accuracy and cross-domain generalization.

Key findings

CS-DFD achieves state-of-the-art performance, surpassing baselines by achieving high accuracy on both in-domain (FF++ ~99% AUC) and cross-domain datasets (average ~88% AUC on Celeb-DF, DFDC, DFDCp, UADFV). The method successfully overcomes the '1+1 < 2' problem by resolving gradient conflicts, enabling effective utilization of diverse training data. Both UVS and CGR modules individually contribute positively, with their combination yielding the best overall performance and generalizability across different backbone architectures.

Approach

The CS-DFD framework resolves gradient conflicts between original and online synthesized forgeries during joint training. It employs an Update Vector Search (UVS) module to find a conflict-free parameter update vector that maximizes the simultaneous reduction of losses from both data types. Additionally, a Conflict Gradient Reduction (CGR) module introduces a Conflict Descent Loss to guide the network in learning a low-conflict feature embedding space, thereby preserving diverse discriminative knowledge.

Datasets

FF++ (Deepfakes, FaceSwap, Face2Face, FaceShifter, NeuralTextures), Celeb-DF, DFDC, DFDCp, UADFV

Model(s)

EfficientNet, Xception, ViT-L, ViT-B

Author countries

China, Singapore

← Previous