FaceForensics++: Learning to Detect Manipulated Facial Images

View on arXiv ← Back to list

Authors: Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, Matthias Nießner

Published: 2019-01-25 16:38:21+00:00

AI Summary

This paper introduces FaceForensics++, a large-scale benchmark dataset (over 1.8 million manipulated images) for facial manipulation detection, exceeding previous datasets by an order of magnitude. The authors demonstrate that using domain-specific knowledge significantly improves forgery detection accuracy, even surpassing human performance.

Abstract

The rapid progress in synthetic image generation and manipulation has now come to a point where it raises significant concerns for the implications towards society. At best, this leads to a loss of trust in digital content, but could potentially cause further harm by spreading false information or fake news. This paper examines the realism of state-of-the-art image manipulations, and how difficult it is to detect them, either automatically or by humans. To standardize the evaluation of detection methods, we propose an automated benchmark for facial manipulation detection. In particular, the benchmark is based on DeepFakes, Face2Face, FaceSwap and NeuralTextures as prominent representatives for facial manipulations at random compression level and size. The benchmark is publicly available and contains a hidden test set as well as a database of over 1.8 million manipulated images. This dataset is over an order of magnitude larger than comparable, publicly available, forgery datasets. Based on this data, we performed a thorough analysis of data-driven forgery detectors. We show that the use of additional domainspecific knowledge improves forgery detection to unprecedented accuracy, even in the presence of strong compression, and clearly outperforms human observers.

Key findings

The proposed XceptionNet model significantly outperforms human observers in detecting facial forgeries, achieving high accuracy even with compressed videos. The results highlight the importance of large-scale datasets and incorporating domain-specific knowledge (face tracking) for improved deepfake detection.

Approach

The authors address the problem by creating a large-scale dataset of manipulated facial images using four state-of-the-art manipulation methods. They then train a deep learning model (XceptionNet) on this dataset for forgery detection, incorporating face tracking to extract relevant image regions.

Datasets

FaceForensics++ dataset (over 1.8 million manipulated images from 1000 videos, generated using DeepFakes, Face2Face, FaceSwap, and NeuralTextures, at various compression levels). The dataset also includes a hidden test set for benchmarking.

Model(s)

XceptionNet (primarily), along with comparative evaluations using other models such as MesoNet, a model by Cozzolino et al., a model by Bayar and Stamm, and a model by Rahmouni et al., and a Support Vector Machine (SVM) with handcrafted steganalysis features.

Author countries

Germany, Italy

← Previous