DeepFake Detection by Analyzing Convolutional Traces

View on arXiv ← Back to list

Authors: Luca Guarnera, Oliver Giudice, Sebastiano Battiato

Published: 2020-04-22 09:02:55+00:00

AI Summary

This paper proposes a novel deepfake detection method that analyzes convolutional traces, essentially fingerprints left by the generative process, in images of human faces. The method uses an Expectation-Maximization (EM) algorithm to extract local features reflecting the underlying convolutional generative process, enabling differentiation between various GAN architectures.

Abstract

The Deepfake phenomenon has become very popular nowadays thanks to the possibility to create incredibly realistic images using deep learning tools, based mainly on ad-hoc Generative Adversarial Networks (GAN). In this work we focus on the analysis of Deepfakes of human faces with the objective of creating a new detection method able to detect a forensics trace hidden in images: a sort of fingerprint left in the image generation process. The proposed technique, by means of an Expectation Maximization (EM) algorithm, extracts a set of local features specifically addressed to model the underlying convolutional generative process. Ad-hoc validation has been employed through experimental tests with naive classifiers on five different architectures (GDWCT, STARGAN, ATTGAN, STYLEGAN, STYLEGAN2) against the CELEBA dataset as ground-truth for non-fakes. Results demonstrated the effectiveness of the technique in distinguishing the different architectures and the corresponding generation process.

Key findings

The method effectively distinguishes between authentic images and deepfakes generated by different GAN architectures, achieving high accuracy. The results highlight the effectiveness of analyzing convolutional traces as a forensic marker for deepfake detection, surpassing the performance of a VGG-16 network used for comparison.

Approach

The approach employs an Expectation-Maximization (EM) algorithm to extract local features from images that model the convolutional generative process used in deepfake creation. These features, representing convolutional traces, are then fed into naive classifiers (K-NN, SVM, LDA) to distinguish between authentic and deepfake images.

Datasets

CELEBA dataset for real images; deepfakes generated using GDWCT, STARGAN, ATTGAN, STYLEGAN, and STYLEGAN2.

Model(s)

K-NN, SVM, LDA classifiers; the deepfake images were generated by GDWCT, STARGAN, ATTGAN, STYLEGAN, and STYLEGAN2 GANs. The EM algorithm was the core of the proposed technique.

Author countries

Italy

← Previous