WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection

View on arXiv ← Back to list

Authors: Bojia Zi, Minghao Chang, Jingjing Chen, Xingjun Ma, Yu-Gang Jiang

Published: 2021-01-05 11:10:32+00:00

AI Summary

This paper introduces WildDeepfake, a new dataset of real-world deepfake videos collected from the internet, addressing the limitations of existing datasets. The authors also propose Attention-based Deepfake Detection Networks (ADDNets) that leverage attention masks for improved detection performance.

Abstract

In recent years, the abuse of a face swap technique called deepfake has raised enormous public concerns. So far, a large number of deepfake videos (known as deepfakes) have been crafted and uploaded to the internet, calling for effective countermeasures. One promising countermeasure against deepfakes is deepfake detection. Several deepfake datasets have been released to support the training and testing of deepfake detectors, such as DeepfakeDetection and FaceForensics++. While this has greatly advanced deepfake detection, most of the real videos in these datasets are filmed with a few volunteer actors in limited scenes, and the fake videos are crafted by researchers using a few popular deepfake softwares. Detectors developed on these datasets may become less effective against real-world deepfakes on the internet. To better support detection against real-world deepfakes, in this paper, we introduce a new dataset WildDeepfake which consists of 7,314 face sequences extracted from 707 deepfake videos collected completely from the internet. WildDeepfake is a small dataset that can be used, in addition to existing datasets, to develop and test the effectiveness of deepfake detectors against real-world deepfakes. We conduct a systematic evaluation of a set of baseline detection networks on both existing and our WildDeepfake datasets, and show that WildDeepfake is indeed a more challenging dataset, where the detection performance can decrease drastically. We also propose two (eg. 2D and 3D) Attention-based Deepfake Detection Networks (ADDNets) to leverage the attention masks on real/fake faces for improved detection. We empirically verify the effectiveness of ADDNets on both existing datasets and WildDeepfake. The dataset is available at: https://github.com/OpenTAI/wild-deepfake.

Key findings

WildDeepfake proved more challenging for existing deepfake detectors than existing datasets, showing a significant drop in performance. The proposed ADDNets achieved comparable or better performance on existing datasets and significantly improved performance on WildDeepfake, particularly the 2D version.

Approach

The authors address the problem by creating a new dataset, WildDeepfake, comprising internet-sourced deepfakes. They propose Attention-based Deepfake Detection Networks (ADDNets), which utilize attention masks generated from facial landmarks to refine feature maps at multiple layers, improving the detection of subtle manipulations in deepfakes.

Datasets

WildDeepfake, DeepfakeDetection, Deepfake-TIMIT (LQ and HQ), FaceForensics++ (LQ and HQ)

Model(s)

ADDNet-2D, ADDNet-3D, AlexNet, VGG16, ResNetV2-50/101/152, Inception-v2, XceptionNet, MesoNet-1, MesoNet-4, MesoNet-Inception, P3D, C3D, I3D

Author countries

China, Australia

← Previous