A Novel Framework for Assessment of Learning-based Detectors in Realistic Conditions with Application to Deepfake Detection

View on arXiv ← Back to list

Authors: Yuhang Lu, Ruizhi Luo, Touradj Ebrahimi

Published: 2022-03-22 15:03:56+00:00

AI Summary

This paper introduces a framework for evaluating the robustness of learning-based detectors, specifically deepfake detectors, under realistic conditions by applying various image processing operations. The framework's effectiveness is demonstrated by designing a data augmentation strategy improving the generalization ability of deepfake detectors.

Abstract

Deep convolutional neural networks have shown remarkable results on multiple detection tasks. Despite the significant progress, the performance of such detectors are often assessed in public benchmarks under non-realistic conditions. Specifically, impact of conventional distortions and processing operations such as compression, noise, and enhancement are not sufficiently studied. This paper proposes a rigorous framework to assess performance of learning-based detectors in more realistic situations. An illustrative example is shown under deepfake detection context. Inspired by the assessment results, a data augmentation strategy based on natural image degradation process is designed, which significantly improves the generalization ability of two deepfake detectors.

Key findings

Real-world image processing operations significantly impact deepfake detection accuracy. A data augmentation strategy based on realistic image degradation improves the robustness of deepfake detectors against these operations, enhancing generalization ability while maintaining performance on unaltered data. The impact of training data quality on detector robustness is also analyzed.

Approach

The authors propose an assessment framework that evaluates deepfake detectors' performance under various real-world image processing operations (compression, noise, smoothing, etc.). Based on the assessment, a data augmentation strategy using natural image degradation processes is developed to improve detector robustness.

Datasets

FaceForensics++, Celeb-DF

Model(s)

Capsule-Forensics, XceptionNet

Author countries

Switzerland

← Previous