EEG-Features for Generalized Deepfake Detection

View on arXiv ← Back to list

Authors: Arian Beckmann, Tilman Stephani, Felix Klotzsche, Yonghao Chen, Simon M. Hofmann, Arno Villringer, Michael Gaebler, Vadim Nikulin, Sebastian Bosse, Peter Eisert, Anna Hilsmann

Published: 2024-05-14 12:06:44+00:00

AI Summary

This research explores using electroencephalography (EEG) data to detect deepfakes. EEG signals, measured while a participant viewed deepfake and real images, were used as features for a support vector classifier to discriminate between real and manipulated images. Preliminary results suggest EEG-based deepfake detection is feasible and may generalize to unseen deepfake methods.

Abstract

Since the advent of Deepfakes in digital media, the development of robust and reliable detection mechanism is urgently called for. In this study, we explore a novel approach to Deepfake detection by utilizing electroencephalography (EEG) measured from the neural processing of a human participant who viewed and categorized Deepfake stimuli from the FaceForensics++ datset. These measurements serve as input features to a binary support vector classifier, trained to discriminate between real and manipulated facial images. We examine whether EEG data can inform Deepfake detection and also if it can provide a generalized representation capable of identifying Deepfakes beyond the training domain. Our preliminary results indicate that human neural processing signals can be successfully integrated into Deepfake detection frameworks and hint at the potential for a generalized neural representation of artifacts in computer generated faces. Moreover, our study provides next steps towards the understanding of how digital realism is embedded in the human cognitive system, possibly enabling the development of more realistic digital avatars in the future.

Key findings

EEG features successfully discriminated between real and deepfake images. The model showed some ability to generalize to deepfake methods not seen during training, indicating a potential for a generalized representation of deepfake artifacts in neural processing. Further research with more participants and deepfake variations is needed.

Approach

The study measured EEG signals from a participant viewing deepfake and real images from FaceForensics++. These EEG features were used to train a binary support vector classifier to distinguish between real and manipulated images. The classifier's performance was evaluated on both seen and unseen deepfake methods.

Datasets

FaceForensics++

Model(s)

Binary Support Vector Classifier (SVC)

Author countries

Germany

← Previous