Deepfake: Definitions, Performance Metrics and Standards, Datasets and Benchmarks, and a Meta-Review

View on arXiv ← Back to list

Authors: Enes Altuncu, Virginia N. L. Franqueira, Shujun Li

Published: 2022-08-21 17:31:31+00:00

AI Summary

This paper provides a comprehensive overview of the deepfake research ecosystem, covering definitions, performance metrics and standards, datasets, challenges, competitions, and benchmarks. It also includes a meta-review of 12 deepfake-related survey papers published in 2020 and 2021, analyzing key challenges and recommendations.

Abstract

Recent advancements in AI, especially deep learning, have contributed to a significant increase in the creation of new realistic-looking synthetic media (video, image, and audio) and manipulation of existing media, which has led to the creation of the new term ``deepfake''. Based on both the research literature and resources in English and in Chinese, this paper gives a comprehensive overview of deepfake, covering multiple important aspects of this emerging concept, including 1) different definitions, 2) commonly used performance metrics and standards, and 3) deepfake-related datasets, challenges, competitions and benchmarks. In addition, the paper also reports a meta-review of 12 selected deepfake-related survey papers published in 2020 and 2021, focusing not only on the mentioned aspects, but also on the analysis of key challenges and recommendations. We believe that this paper is the most comprehensive review of deepfake in terms of aspects covered, and the first one covering both the English and Chinese literature and sources.

Key findings

The review highlights a lack of consensus on the definition of "deepfake." It finds that existing deepfake datasets vary significantly in quality and realness. The meta-review reveals a focus on improving the robustness, scalability, generalizability, and explainability of deepfake detection methods as a key challenge and future research direction.

Approach

The authors conduct a systematic review of existing literature in English and Chinese, focusing on deepfake definitions, evaluation metrics, datasets, challenges, and benchmarks. They also perform a meta-review of 12 survey papers to identify key challenges and future research directions.

Datasets

Many datasets are mentioned and categorized into image, video, audio, and hybrid categories. Examples include FaceForensics++, DeepfakeTIMIT, DFDC, ASVspoof 2019/2021, ForgeryNet, and several others. Details on sizes and characteristics of each are provided in the paper.

Model(s)

The paper reviews models and architectures used in various deepfake detection challenges and benchmarks but does not focus on specific models used by the authors themselves. Mentioned architectures include MesoNet, EfficientNet, and others.

Author countries

← Previous