FaceForensics++: Learning to Detect Manipulated Facial Images

Authors: Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, Matthias Nießner

Published: 2019-01-25 16:38:21+00:00

Comment: Video: https://youtu.be/x2g48Q2I2ZQ

AI Summary

This paper introduces FaceForensics++, a large-scale benchmark dataset comprising over 1.8 million manipulated facial images generated from four state-of-the-art methods, along with an automated benchmark for facial manipulation detection. It performs a thorough analysis of data-driven forgery detectors, demonstrating that incorporating domain-specific knowledge significantly improves detection accuracy, even under strong compression, and clearly outperforms human observers.

Abstract

The rapid progress in synthetic image generation and manipulation has now come to a point where it raises significant concerns for the implications towards society. At best, this leads to a loss of trust in digital content, but could potentially cause further harm by spreading false information or fake news. This paper examines the realism of state-of-the-art image manipulations, and how difficult it is to detect them, either automatically or by humans. To standardize the evaluation of detection methods, we propose an automated benchmark for facial manipulation detection. In particular, the benchmark is based on DeepFakes, Face2Face, FaceSwap and NeuralTextures as prominent representatives for facial manipulations at random compression level and size. The benchmark is publicly available and contains a hidden test set as well as a database of over 1.8 million manipulated images. This dataset is over an order of magnitude larger than comparable, publicly available, forgery datasets. Based on this data, we performed a thorough analysis of data-driven forgery detectors. We show that the use of additional domainspecific knowledge improves forgery detection to unprecedented accuracy, even in the presence of strong compression, and clearly outperforms human observers.

Key findings

Automated data-driven forgery detectors, particularly those leveraging domain-specific face tracking and large-capacity CNNs like XceptionNet, achieve unprecedented accuracy in detecting facial manipulations, significantly outperforming human observers. The detection performance is highly dependent on video quality and training corpus size, with larger datasets being crucial for robust detection in low-quality, compressed scenarios. GAN-based manipulations like NeuralTextures were found to be particularly challenging to detect.

Approach

The authors create a large-scale dataset, FaceForensics++, by applying four prominent facial manipulation methods (DeepFakes, Face2Face, FaceSwap, NeuralTextures) to 1,000 pristine YouTube videos, generating over 1.8 million manipulated images at various compression levels. They then evaluate several deep-learning-based and hand-crafted forgery detectors, proposing a pipeline that incorporates robust face tracking and utilizes a CNN (XceptionNet) for classification.

Datasets

FaceForensics++, FaceForensics (preliminary version), YouTube-8m (for pristine video acquisition).

Model(s)

UNKNOWN

Author countries

Germany, Italy

← Previous