Deepfake detection by exploiting surface anomalies: the SurFake approach

View on arXiv ← Back to list

Authors: Andrea Ciamarra, Roberto Caldelli, Federico Becattini, Lorenzo Seidenari, Alberto Del Bimbo

Published: 2023-10-31 16:54:14+00:00

AI Summary

This paper introduces SurFake, a deepfake detection approach that leverages surface anomalies in images. It analyzes the impact of deepfake generation on scene geometry and acquisition characteristics, using a Global Surface Descriptor (GSD) to train a CNN for deepfake detection.

Abstract

The ever-increasing use of synthetically generated content in different sectors of our everyday life, one for all media information, poses a strong need for deepfake detection tools in order to avoid the proliferation of altered messages. The process to identify manipulated content, in particular images and videos, is basically performed by looking for the presence of some inconsistencies and/or anomalies specifically due to the fake generation process. Different techniques exist in the scientific literature that exploit diverse ad-hoc features in order to highlight possible modifications. In this paper, we propose to investigate how deepfake creation can impact on the characteristics that the whole scene had at the time of the acquisition. In particular, when an image (video) is captured the overall geometry of the scene (e.g. surfaces) and the acquisition process (e.g. illumination) determine a univocal environment that is directly represented by the image pixel values; all these intrinsic relations are possibly changed by the deepfake generation process. By resorting to the analysis of the characteristics of the surfaces depicted in the image it is possible to obtain a descriptor usable to train a CNN for deepfake detection: we refer to such an approach as SurFake. Experimental results carried out on the FF++ dataset for different kinds of deepfake forgeries and diverse deep learning models confirm that such a feature can be adopted to discriminate between pristine and altered images; furthermore, experiments witness that it can also be combined with visual data to provide a certain improvement in terms of detection accuracy.

Key findings

Experiments on FF++ demonstrate that the GSD feature alone achieves around 75% accuracy. Combining GSD with RGB data improves detection accuracy, particularly for challenging forgeries like NeuralTextures. EfficientNet-B0 and MobileNetV2 show superior performance.

Approach

SurFake extracts a Global Surface Descriptor (GSD) from face crops using UpRightNet, capturing surface geometry information. This GSD, along with the original RGB image, is then fed into a CNN classifier to distinguish between real and fake images.

Datasets

FaceForensics++ (FF++) dataset, including DeepFakes (DF), Face2Face (F2F), FaceShifter (FSH), FaceSwap (FS), and NeuralTextures (NT) forgeries.

Model(s)

ResNet50, MobileNetV2, EfficientNet-B0, and Xception CNN architectures. UpRightNet is used for surface geometry feature extraction.

Author countries

Italy

← Previous