Physiological Signals as a Forensic Modality for Talking-Face Deepfake Detection

Authors: Othmane Harraq, Tamer Aldwairi

Published: 2026-07-23 19:47:15+00:00

AI Summary

This research proposes a deepfake detection framework for talking-face (TF) videos by analyzing remote photoplethysmography (rPPG) waveforms, which are physiological signals representing cardiac pulse. The method extracts rPPG waveforms using RhythmFormer and classifies them with lightweight 1D classifiers, achieving competitive performance on a subject-independent protocol, specifically on the TF subset of Celeb-DF++. A key finding is that detection difficulty strongly depends on the specific TF generation method's pixel synthesis strategy.

Abstract

Talking-face (TF) deepfake generation synthesizes photore- alistic facial video from a static source image and an au- dio signal, producing forgeries that current image-based detectors consistently fail to identify. Unlike face-swap ma- nipulation, TF synthesis has no underlying real video from which to inherit physiological characteristics, making re- mote photoplethysmography (rPPG) a uniquely motivated detection modality for this forgery category. We propose a detection framework that extracts per-video rPPG wave- forms via RhythmFormer and trains a suite of lightweight classifiers to distinguish real from synthesized physiologi- cal signals. Evaluated on the TF subset of Celeb-DF++ un- der a strict subject-independent protocol, where test identi- ties are completely separated from training identities, our 1D ResNet achieves an AUC of 0.806 and EER of 27.8%, placing it within 2.4 points of the best published general- purpose detector (Effort, ICML 2025) while operating ex- clusively on the physiological channel. We document a con- trolled reproduction study of DeepFakesON-Phys, the rep- resentative prior rPPG detector, demonstrating degrada- tion from AUC 0.999 on legacy face-swap data to 0.622 on the TF subset of Celeb-DF++. We further show that detec- tion difficulty is strongly method-dependent: AUC ranges from 0.985 (Real3DPortrait) to 0.690 (IP-LAP) across the seven TF generators, with the ranking remaining perfectly stable across all evaluation protocols. This spread reflects an interpretable physiological property of each generator rather than evaluation noise, and constitutes the primary theoretical contribution of the work.

Key findings

The rPPG-based detector achieved an AUC of 0.806, demonstrating its effectiveness for TF deepfake detection using only physiological signals, bridging a gap to state-of-the-art general-purpose detectors. Detection difficulty is highly dependent on the TF generation method, with AUC ranging from 0.985 for Real3DPortrait (no coherent rPPG) to 0.690 for IP-LAP (physiologically plausible rPPG), highlighting that methods preserving identity structure are harder to detect.

Approach

The approach extracts per-video rPPG waveforms from the visual signal of talking-face videos using RhythmFormer. These extracted waveforms are then fed into lightweight 1D classifiers, primarily a 1D ResNet, to distinguish between real and synthesized physiological signals. The training uses a weighted binary cross-entropy loss and data augmentation to prevent overfitting to identity-specific characteristics.

Datasets

Celeb-DF++ (specifically its Talking-Face subset)

Model(s)

RhythmFormer (for rPPG extraction), 1D ResNet, 1D CNN, 1D Transformer (for classification)

Author countries

United States

← Previous