Assessment Framework for Deepfake Detection in Real-world Situations

View on arXiv ← Back to list

Authors: Yuhang Lu, Touradj Ebrahimi

Published: 2023-04-12 19:09:22+00:00

AI Summary

This paper introduces a novel assessment framework for evaluating deepfake detection methods under realistic conditions, considering various image and video processing operations. The framework quantifies the robustness of detectors to these operations and proposes a stochastic degradation-based data augmentation method to improve their performance.

Abstract

Detecting digital face manipulation in images and video has attracted extensive attention due to the potential risk to public trust. To counteract the malicious usage of such techniques, deep learning-based deepfake detection methods have been employed and have exhibited remarkable performance. However, the performance of such detectors is often assessed on related benchmarks that hardly reflect real-world situations. For example, the impact of various image and video processing operations and typical workflow distortions on detection accuracy has not been systematically measured. In this paper, a more reliable assessment framework is proposed to evaluate the performance of learning-based deepfake detectors in more realistic settings. To the best of our acknowledgment, it is the first systematic assessment approach for deepfake detectors that not only reports the general performance under real-world conditions but also quantitatively measures their robustness toward different processing operations. To demonstrate the effectiveness and usage of the framework, extensive experiments and detailed analysis of three popular deepfake detection methods are further presented in this paper. In addition, a stochastic degradation-based data augmentation method driven by realistic processing operations is designed, which significantly improves the robustness of deepfake detectors.

Key findings

Even mild real-world processing significantly impacts deepfake detection accuracy. The proposed stochastic degradation-based augmentation method substantially improves the robustness of detectors across various distortions, including those not explicitly included in the augmentation process. The framework highlights the limitations of existing detectors and provides valuable insights for future research.

Approach

The authors propose an assessment framework that systematically evaluates deepfake detectors by applying various image and video processing operations (noise, compression, resizing, etc.) to test data at different severity levels. The framework measures the impact of these operations on detection accuracy and uses the average performance across all operations as an overall score. They also introduce a stochastic data augmentation method to improve robustness by mimicking real-world degradation.

Datasets

FaceForensics++ (FFpp), Celeb-DFv2

Model(s)

Capsule-Forensics, XceptionNet, SBIs (Self-blended Images)

Author countries

Switzerland

← Previous