Quality-Agnostic Deepfake Detection with Intra-model Collaborative Learning

View on arXiv ← Back to list

Authors: Binh M. Le, Simon S. Woo

Published: 2023-09-12 02:01:31+00:00

AI Summary

This paper proposes QAD, a quality-agnostic deepfake detection method using an intra-model collaborative learning framework. QAD simultaneously detects deepfakes of varying qualities by maximizing the dependency between intermediate representations of different quality levels via Hilbert-Schmidt Independence Criterion (HSIC) and enhancing robustness against image corruption using an Adversarial Weight Perturbation (AWP) module.

Abstract

Deepfake has recently raised a plethora of societal concerns over its possible security threats and dissemination of fake information. Much research on deepfake detection has been undertaken. However, detecting low quality as well as simultaneously detecting different qualities of deepfakes still remains a grave challenge. Most SOTA approaches are limited by using a single specific model for detecting certain deepfake video quality type. When constructing multiple models with prior information about video quality, this kind of strategy incurs significant computational cost, as well as model and training data overhead. Further, it cannot be scalable and practical to deploy in real-world settings. In this work, we propose a universal intra-model collaborative learning framework to enable the effective and simultaneous detection of different quality of deepfakes. That is, our approach is the quality-agnostic deepfake detection method, dubbed QAD . In particular, by observing the upper bound of general error expectation, we maximize the dependency between intermediate representations of images from different quality levels via Hilbert-Schmidt Independence Criterion. In addition, an Adversarial Weight Perturbation module is carefully devised to enable the model to be more robust against image corruption while boosting the overall model's performance. Extensive experiments over seven popular deepfake datasets demonstrate the superiority of our QAD model over prior SOTA benchmarks.

Key findings

Extensive experiments on seven deepfake datasets demonstrate QAD's superiority over state-of-the-art methods in quality-agnostic settings. QAD achieves significant improvements, especially for low-quality deepfakes, while requiring fewer computational parameters than multi-model approaches. The ablation studies confirm the effectiveness of both HSIC and AWP modules.

Approach

QAD uses an intra-model collaborative learning framework to simultaneously learn representations from different quality levels of deepfake videos. It maximizes the dependency between these representations using HSIC and improves robustness to image corruption with AWP. This allows for effective detection of deepfakes regardless of quality in a single model.

Datasets

NeuralTextures, Deepfakes, Face2Face, FaceSwap, FaceShifter, CelebDFv2, Face Forensics in the Wild (FFIW10K)

Model(s)

ResNet-50, EfficientNet-B1

Author countries

South Korea

← Previous