DomainForensics: Exposing Face Forgery across Domains via Bi-directional Adaptation

View on arXiv ← Back to list

Authors: Qingxuan Lv, Yuezun Li, Junyu Dong, Sheng Chen, Hui Yu, Huiyu Zhou, Shu Zhang

Published: 2023-12-17 10:46:46+00:00

AI Summary

DomainForensics addresses the challenge of DeepFake detection's poor performance on unseen forgeries by employing unsupervised domain adaptation. It uses a novel bi-directional adaptation strategy, transferring forgery knowledge from known to new forgeries and vice-versa, leveraging adversarial training and self-distillation to capture subtle forgery traces.

Abstract

Recent DeepFake detection methods have shown excellent performance on public datasets but are significantly degraded on new forgeries. Solving this problem is important, as new forgeries emerge daily with the continuously evolving generative techniques. Many efforts have been made for this issue by seeking the commonly existing traces empirically on data level. In this paper, we rethink this problem and propose a new solution from the unsupervised domain adaptation perspective. Our solution, called DomainForensics, aims to transfer the forgery knowledge from known forgeries to new forgeries. Unlike recent efforts, our solution does not focus on data view but on learning strategies of DeepFake detectors to capture the knowledge of new forgeries through the alignment of domain discrepancies. In particular, unlike the general domain adaptation methods which consider the knowledge transfer in the semantic class category, thus having limited application, our approach captures the subtle forgery traces. We describe a new bi-directional adaptation strategy dedicated to capturing the forgery knowledge across domains. Specifically, our strategy considers both forward and backward adaptation, to transfer the forgery knowledge from the source domain to the target domain in forward adaptation and then reverse the adaptation from the target domain to the source domain in backward adaptation. In forward adaptation, we perform supervised training for the DeepFake detector in the source domain and jointly employ adversarial feature adaptation to transfer the ability to detect manipulated faces from known forgeries to new forgeries. In backward adaptation, we further improve the knowledge transfer by coupling adversarial adaptation with self-distillation on new forgeries. This enables the detector to expose new forgery features from unlabeled data and avoid forgetting the known knowledge of known...

Key findings

DomainForensics significantly outperforms state-of-the-art methods in cross-domain DeepFake detection scenarios (cross-manipulation methods, datasets, and types). The bi-directional adaptation strategy proves highly effective, and the framework is shown to be plug-and-play, adaptable to various architectures and improving the generalization of existing methods.

Approach

DomainForensics formulates DeepFake detection as an unsupervised domain adaptation problem. It uses a bi-directional adaptation strategy with forward adaptation (supervised training on known forgeries and adversarial feature adaptation) and backward adaptation (self-distillation on new forgeries and adversarial adaptation). This allows the model to learn common forgery features across domains.

Datasets

FaceForensics++ (FF++), Celeb-DF, StyleGAN, DFDCP, FFIW

Model(s)

Vision Transformer (ViT), ResNet, Xception, EfficientNet (used in ablation study)

Author countries

China, U.K.

← Previous