DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection

View on arXiv ← Back to list

Authors: Ruben Tolosana, Ruben Vera-Rodriguez, Julian Fierrez, Aythami Morales, Javier Ortega-Garcia

Published: 2020-01-01 09:54:34+00:00

AI Summary

This survey paper comprehensively reviews face manipulation techniques (including DeepFakes) and their detection methods. It categorizes manipulations into four types, examining their techniques, available datasets, and benchmark results for detection.

Abstract

The free access to large-scale public databases, together with the fast progress of deep learning techniques, in particular Generative Adversarial Networks, have led to the generation of very realistic fake content with its corresponding implications towards society in this era of fake news. This survey provides a thorough review of techniques for manipulating face images including DeepFake methods, and methods to detect such manipulations. In particular, four types of facial manipulation are reviewed: i) entire face synthesis, ii) identity swap (DeepFakes), iii) attribute manipulation, and iv) expression swap. For each manipulation group, we provide details regarding manipulation techniques, existing public databases, and key benchmarks for technology evaluation of fake detection methods, including a summary of results from those evaluations. Among all the aspects discussed in the survey, we pay special attention to the latest generation of DeepFakes, highlighting its improvements and challenges for fake detection. In addition to the survey information, we also discuss open issues and future trends that should be considered to advance in the field.

Key findings

While many methods achieve high accuracy in controlled settings, they often exhibit poor generalization to unseen datasets and real-world scenarios with varying compression and noise levels. The latest generation of DeepFakes pose a significant challenge to current detection methods, with lower detection accuracy compared to earlier methods.

Approach

The paper conducts a comprehensive survey of existing literature, categorizing face manipulation techniques and reviewing datasets and detection methods for each category. It analyzes the performance of various detection approaches across different datasets and highlights the challenges in detecting newer, more realistic DeepFakes.

Datasets

UADFV, DeepfakeTIMIT, FaceForensics++, DeepFakeDetection, Celeb-DF, DFDC Preview, 100K-Generated-Images, 100K-Faces, DFFD, iFakeFaceDB, CelebA, FFHQ, CASIA-WebFace, VGGFace2, various datasets collected online

Model(s)

Various CNN architectures (e.g., VGG16, ResNet, XceptionNet), RNNs (LSTMs), SVM, Autoencoders, Capsule Networks, and attention mechanisms are mentioned as models used in different studies for deepfake detection. Specific models used in each study are detailed within the paper.

Author countries

Spain

← Previous