Unearthing Common Inconsistency for Generalisable Deepfake Detection

View on arXiv ← Back to list

Authors: Beilin Chu, Xuan Xu, Weike You, Linna Zhou

Published: 2023-11-20 06:04:09+00:00

AI Summary

This paper proposes a novel deepfake video detection method, Unearthing Common Inconsistency (UCI), that focuses on capturing frame inconsistencies common across different forgery techniques. UCI utilizes self-supervised contrastive learning and a temporally-preserved augmentation module to enhance the model's focus on temporal information and improve generalization across unseen domains.

Abstract

Deepfake has emerged for several years, yet efficient detection techniques could generalize over different manipulation methods require further research. While current image-level detection method fails to generalize to unseen domains, owing to the domain-shift phenomenon brought by CNN's strong inductive bias towards Deepfake texture, video-level one shows its potential to have both generalization across multiple domains and robustness to compression. We argue that although distinct face manipulation tools have different inherent bias, they all disrupt the consistency between frames, which is a natural characteristic shared by authentic videos. Inspired by this, we proposed a detection approach by capturing frame inconsistency that broadly exists in different forgery techniques, termed unearthing-common-inconsistency (UCI). Concretely, the UCI network based on self-supervised contrastive learning can better distinguish temporal consistency between real and fake videos from multiple domains. We introduced a temporally-preserved module method to introduce spatial noise perturbations, directing the model's attention towards temporal information. Subsequently, leveraging a multi-view cross-correlation learning module, we extensively learn the disparities in temporal representations between genuine and fake samples. Extensive experiments demonstrate the generalization ability of our method on unseen Deepfake domains.

Key findings

The UCI method outperforms several baselines in cross-domain generalization tests on FaceForensics++. It achieves the highest average AUC score across Celeb-DF, DFDC-preview, and FaceShifter datasets after training only on FaceForensics++, demonstrating strong generalization capabilities. Ablation studies confirm the effectiveness of the proposed augmentation and contrastive learning strategies.

Approach

The UCI method uses a 3D convolutional network to extract temporal consistency representations. It employs a contrastive learning strategy to distinguish between real and fake videos and a temporally-preserved augmentation to focus on temporal information, improving generalization.

Datasets

FaceForensics++, Celeb-DF, DFDC-preview, FaceShifter

Model(s)

Inflated 3D ConvNet (I3D), with LSTM and C3D used in ablation studies.

Author countries

China

← Previous