Quality-Agnostic Deepfake Detection with Intra-model Collaborative Learning

Authors: Binh M. Le, Simon S. Woo

Published: 2023-09-12 02:01:31+00:00

Journal Ref: International Conference on Computer Vision 2023

AI Summary

This paper proposes Quality-Agnostic Deepfake detection (QAD), an intra-model collaborative learning framework designed to simultaneously detect deepfakes across various quality levels, including low-quality ones. QAD employs Hilbert-Schmidt Independence Criterion (HSIC) to maximize dependency between intermediate representations of different quality images and an Adversarial Weight Perturbation (AWP) module for robustness against image corruption. Extensive experiments demonstrate QAD's superior performance over prior state-of-the-art benchmarks on multiple deepfake datasets.

Abstract

Deepfake has recently raised a plethora of societal concerns over its possible security threats and dissemination of fake information. Much research on deepfake detection has been undertaken. However, detecting low quality as well as simultaneously detecting different qualities of deepfakes still remains a grave challenge. Most SOTA approaches are limited by using a single specific model for detecting certain deepfake video quality type. When constructing multiple models with prior information about video quality, this kind of strategy incurs significant computational cost, as well as model and training data overhead. Further, it cannot be scalable and practical to deploy in real-world settings. In this work, we propose a universal intra-model collaborative learning framework to enable the effective and simultaneous detection of different quality of deepfakes. That is, our approach is the quality-agnostic deepfake detection method, dubbed QAD . In particular, by observing the upper bound of general error expectation, we maximize the dependency between intermediate representations of images from different quality levels via Hilbert-Schmidt Independence Criterion. In addition, an Adversarial Weight Perturbation module is carefully devised to enable the model to be more robust against image corruption while boosting the overall model's performance. Extensive experiments over seven popular deepfake datasets demonstrate the superiority of our QAD model over prior SOTA benchmarks.

Key findings

QAD consistently outperforms prior state-of-the-art benchmarks across seven popular deepfake datasets in quality-agnostic settings, demonstrating significant improvements in AUC (e.g., up to 5.28% for NeuralTextures). It also achieves new state-of-the-art performance in quality-aware settings with fewer parameters and without requiring prior knowledge of input quality. Ablation studies confirm the effectiveness of HSIC and AWP, showing improved robustness and more consistent feature representations.

Approach

The QAD approach utilizes an intra-model collaborative learning framework. It minimizes the discrepancy between raw and compressed image representations by maximizing the dependency between their intermediate features using the Hilbert-Schmidt Independence Criterion (HSIC). Additionally, an Adversarial Weight Perturbation (AWP) module is applied to the model's parameters to enhance robustness against varying input image compressions.

Datasets

NeuralTextures (NT), Deepfakes (DF), Face2Face (F2F), FaceSwap (FS), FaceShifter (FSH), CelebDFV2 (CDFv2), Face Forensics in the Wild (FFIW10K), DFDC, WildDeepfake

Model(s)

RESNET-50 (QAD-R), EFFICIENTNET-B1 (QAD-E)

Author countries

South Korea

← Previous