Media Forensics and DeepFakes: an overview

View on arXiv ← Back to list

Authors: Luisa Verdoliva

Published: 2020-01-18 00:13:32+00:00

AI Summary

This review paper analyzes methods for visual media integrity verification, focusing on deepfake detection. It highlights the limitations of current forensic tools, key challenges, and future research directions in the field.

Abstract

With the rapid progress of recent years, techniques that generate and manipulate multimedia content can now guarantee a very advanced level of realism. The boundary between real and synthetic media has become very thin. On the one hand, this opens the door to a series of exciting applications in different fields such as creative arts, advertising, film production, video games. On the other hand, it poses enormous security threats. Software packages freely available on the web allow any individual, without special skills, to create very realistic fake images and videos. So-called deepfakes can be used to manipulate public opinion during elections, commit fraud, discredit or blackmail people. Potential abuses are limited only by human imagination. Therefore, there is an urgent need for automated tools capable of detecting false multimedia content and avoiding the spread of dangerous false information. This review paper aims to present an analysis of the methods for visual media integrity verification, that is, the detection of manipulated images and videos. Special emphasis will be placed on the emerging phenomenon of deepfakes and, from the point of view of the forensic analyst, on modern data-driven forensic methods. The analysis will help to highlight the limits of current forensic tools, the most relevant issues, the upcoming challenges, and suggest future directions for research.

Key findings

Deep learning methods significantly improve performance over conventional methods, especially under challenging conditions such as compression. However, these methods often suffer from overfitting and lack generalization ability. One-class methods show promise for dealing with unseen manipulations and real-world conditions.

Approach

The paper provides a comprehensive review of existing approaches to multimedia forensics, including conventional methods (based on camera and editing artifacts) and deep learning-based methods for detecting general manipulations and deepfakes. It categorizes these approaches and discusses their strengths and weaknesses.

Datasets

Columbia (color and gray), Casia (v1 and v2), DSO-1, Realistic Tampering Dataset, Wild Web Dataset, various copy-move datasets, double JPEG compression datasets, NIST datasets (NC2016, NC2017, MFC2018, MFC2019), DEFACTO, PS-Battles Dataset, FaceSwap dataset, GAN image datasets, DF-TIMIT, Fake Face in the Wild (FFW), FaceForensics++, DDD, Celeb-DF, DFDC-preview, DFDC, DeeperForensics-1.0, Dresden image database, RAISE dataset, 2018 Kaggle camera model identification dataset, dataset with SDR and HDR images, VISION dataset, SOCRATES dataset, video-ACID database.

Model(s)

Various CNN architectures (including ResNet, U-Net, Siamese networks, Inception modules, dilated convolutions), autoencoders, LSTM networks, capsule networks, SVMs.

Author countries

Italy

← Previous