The Creation and Detection of Deepfakes: A Survey

View on arXiv ← Back to list

Authors: Yisroel Mirsky, Wenke Lee

Published: 2020-04-23 13:35:49+00:00

AI Summary

This survey paper explores the creation and detection of deepfakes, providing a comprehensive overview of the underlying architectures and techniques. It examines current advancements, shortcomings of existing detection solutions, and areas requiring further research.

Abstract

Generative deep learning algorithms have progressed to a point where it is difficult to tell the difference between what is real and what is fake. In 2018, it was discovered how easy it is to use this technology for unethical and malicious applications, such as the spread of misinformation, impersonation of political leaders, and the defamation of innocent individuals. Since then, these `deepfakes' have advanced significantly. In this paper, we explore the creation and detection of deepfakes and provide an in-depth view of how these architectures work. The purpose of this survey is to provide the reader with a deeper understanding of (1) how deepfakes are created and detected, (2) the current trends and advancements in this domain, (3) the shortcomings of the current defense solutions, and (4) the areas which require further research and attention.

Key findings

The survey reveals a rapid advancement in deepfake technology, particularly in identity-agnostic models and high-resolution deepfakes. Existing detection methods face challenges due to the evolving nature of deepfake creation techniques and the potential for adversarial attacks. The authors emphasize the need for proactive, out-of-band solutions beyond content-based detection, such as establishing data provenance frameworks.

Approach

The paper conducts a systematic survey of deepfake creation and detection methods, categorizing them by modality (audio, video, or both) and generalization capability (one-to-one, many-to-one, many-to-many). It analyzes the architectures, datasets used, and key challenges in each category.

Datasets

DeepfakeTIMIT, DFFD, FaceForensics, FaceForensics++, FFW, Celeb-DF, and other custom deepfake datasets are mentioned.

Model(s)

A wide variety of models are discussed, including Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and combinations thereof. Specific models mentioned include CycleGAN, pix2pix, pix2pixHD, and StyleGAN.

Author countries

USA, Israel

← Previous