DeepFidelity: Perceptual Forgery Fidelity Assessment for Deepfake Detection

View on arXiv ← Back to list

Authors: Chunlei Peng, Huiqing Guo, Decheng Liu, Nannan Wang, Ruimin Hu, Xinbo Gao

Published: 2023-12-07 07:19:45+00:00

AI Summary

DeepFidelity is a novel deepfake detection framework that improves detection accuracy by considering the perceptual forgery fidelity of face images. It maps real and fake faces of varying qualities to distinct scores for more precise classification and uses a Symmetric Spatial Attention Augmentation based vision Transformer (SSAAFormer) for feature extraction.

Abstract

Deepfake detection refers to detecting artificially generated or edited faces in images or videos, which plays an essential role in visual information security. Despite promising progress in recent years, Deepfake detection remains a challenging problem due to the complexity and variability of face forgery techniques. Existing Deepfake detection methods are often devoted to extracting features by designing sophisticated networks but ignore the influence of perceptual quality of faces. Considering the complexity of the quality distribution of both real and fake faces, we propose a novel Deepfake detection framework named DeepFidelity to adaptively distinguish real and fake faces with varying image quality by mining the perceptual forgery fidelity of face images. Specifically, we improve the model's ability to identify complex samples by mapping real and fake face data of different qualities to different scores to distinguish them in a more detailed way. In addition, we propose a network structure called Symmetric Spatial Attention Augmentation based vision Transformer (SSAAFormer), which uses the symmetry of face images to promote the network to model the geographic long-distance relationship at the shallow level and augment local features. Extensive experiments on multiple benchmark datasets demonstrate the superiority of the proposed method over state-of-the-art methods.

Key findings

DeepFidelity outperforms state-of-the-art methods on multiple benchmark datasets, achieving superior accuracy and AUC scores. The approach shows strong generalization ability in cross-dataset evaluations, demonstrating robustness to varying image qualities and forgery techniques. Ablation studies confirm the effectiveness of both the perceptual forgery fidelity assessment and the SSAAFormer architecture.

Approach

DeepFidelity assesses perceptual forgery fidelity by mapping real and fake faces with varying image quality to different scores. It uses a novel architecture, SSAAFormer, which leverages face image symmetry to enhance feature extraction at the shallow network level, combining this with self-attention for global dependencies.

Datasets

FaceForensics++, Celeb-DF (v2), WildDeepfake

Model(s)

SSAAFormer (a vision Transformer with Symmetric Spatial Attention Augmentation), Support Vector Regression (SVR) with RBF kernel

Author countries

China

← Previous