Deepfakes Generation and Detection: State-of-the-art, open challenges, countermeasures, and way forward

View on arXiv ← Back to list

Authors: Momina Masood, Marriam Nawaz, Khalid Mahmood Malik, Ali Javed, Aun Irtaza

Published: 2021-02-25 18:26:50+00:00

AI Summary

This paper provides a comprehensive review of deepfake generation and detection methods for both audio and video modalities. It analyzes existing tools, machine learning approaches, datasets, and evaluation standards, highlighting current limitations and future research directions.

Abstract

Easy access to audio-visual content on social media, combined with the availability of modern tools such as Tensorflow or Keras, open-source trained models, and economical computing infrastructure, and the rapid evolution of deep-learning (DL) methods, especially Generative Adversarial Networks (GAN), have made it possible to generate deepfakes to disseminate disinformation, revenge porn, financial frauds, hoaxes, and to disrupt government functioning. The existing surveys have mainly focused on the detection of deepfake images and videos. This paper provides a comprehensive review and detailed analysis of existing tools and machine learning (ML) based approaches for deepfake generation and the methodologies used to detect such manipulations for both audio and visual deepfakes. For each category of deepfake, we discuss information related to manipulation approaches, current public datasets, and key standards for the performance evaluation of deepfake detection techniques along with their results. Additionally, we also discuss open challenges and enumerate future directions to guide future researchers on issues that need to be considered to improve the domains of both deepfake generation and detection. This work is expected to assist the readers in understanding the creation and detection mechanisms of deepfakes, along with their current limitations and future direction.

Key findings

The survey reveals that deepfake generation techniques have significantly advanced, making them increasingly difficult to detect. While deep learning-based detection methods show promising results, they face challenges related to generalization, explainability, and robustness against adversarial attacks. The need for larger, more diverse, and realistic datasets is also emphasized.

Approach

The authors conducted a systematic review of existing literature on deepfake generation and detection, categorizing techniques by modality (audio, video) and manipulation type (face swap, lip-sync, etc.). They analyzed the approaches, datasets used, and performance evaluation metrics, identifying gaps and future research directions.

Datasets

UADFV, DeepfakeTIMIT, FaceForensics++, Celeb-DF, Deepfake Detection Challenge (DFDC), DeeperForensics (DF), WildDeepfake, LJ Speech, M-AILabs dataset, Mozilla TTS, ASV spoof 2019, Fake-or-Real (FOR) dataset, Baidu Dataset

Model(s)

Various CNNs (ResNet, VGG, etc.), RNNs (LSTM, GRU), GANs, VAEs, Support Vector Machines (SVM), Random Forests (RF), and other deep learning architectures are mentioned in the context of deepfake detection. Specific model names are scattered throughout the paper for different tasks.

Author countries

Pakistan, USA

← Previous