Fighting Malicious Media Data: A Survey on Tampering Detection and Deepfake Detection

View on arXiv ← Back to list

Authors: Junke Wang, Zhenxin Li, Chao Zhang, Jingjing Chen, Zuxuan Wu, Larry S. Davis, Yu-Gang Jiang

Published: 2022-12-12 02:54:08+00:00

AI Summary

This paper provides a comprehensive survey of current media tampering detection approaches, focusing on both image tampering and deepfake detection. It systematically summarizes the newest advances in both fields, highlighting their shared properties and suggesting future research directions.

Abstract

Online media data, in the forms of images and videos, are becoming mainstream communication channels. However, recent advances in deep learning, particularly deep generative models, open the doors for producing perceptually convincing images and videos at a low cost, which not only poses a serious threat to the trustworthiness of digital information but also has severe societal implications. This motivates a growing interest of research in media tampering detection, i.e., using deep learning techniques to examine whether media data have been maliciously manipulated. Depending on the content of the targeted images, media forgery could be divided into image tampering and Deepfake techniques. The former typically moves or erases the visual elements in ordinary images, while the latter manipulates the expressions and even the identity of human faces. Accordingly, the means of defense include image tampering detection and Deepfake detection, which share a wide variety of properties. In this paper, we provide a comprehensive review of the current media tampering detection approaches, and discuss the challenges and trends in this field for future research.

Key findings

The survey reveals significant progress in both tampering and deepfake detection, but highlights challenges in generalization, robustness to post-processing, model attribution, and interpretability. Multimodal approaches show promise for improved detection accuracy.

Approach

The paper conducts a survey of existing research on media tampering and deepfake detection. It categorizes methods by modality (image, video, audio-visual), technique (rule-based, learning-based, hybrid), and manipulation type. It analyzes the strengths and limitations of different approaches.

Datasets

Columbia Gray, Columbia Color, MICC-F8multi, MICC-F220, MICC-F600, MICC-F2000, VIPP Synth., VIPP Real., CoMoFod, CASIA V1.0, CASIA V2.0, Wild Web, NC2016, NC2017, MFC2018, MFC2019, PS-Battles, DEFACTO, IMD2020, ImageForensicsOSN, UADFV, DF-TIMIT-LQ, DF-TIMIT-HQ, DFD, Celeb-DF, FF++, DFFD, DeeperForensics-1.0, DFDC, ForgeryNet

Model(s)

Various CNN architectures, GANs, Autoencoders, LSTMs, Transformers, and other deep learning models are mentioned in relation to existing Deepfake detection methods, but no single model is the main contribution.

Author countries

China, USA

← Previous