Detecting Deepfake Talking Heads from Facial Biometric Anomalies

View on arXiv ← Back to list

Authors: Justin D. Norman, Hany Farid

Published: 2025-07-11 16:29:25+00:00

AI Summary

This paper proposes a novel deepfake detection technique that leverages unnatural patterns in facial biometrics from video. The method analyzes the distribution of facial biometric similarities over time to identify anomalies characteristic of deepfakes, achieving high accuracy across various deepfake generation methods.

Abstract

The combination of highly realistic voice cloning, along with visually compelling avatar, face-swap, or lip-sync deepfake video generation, makes it relatively easy to create a video of anyone saying anything. Today, such deepfake impersonations are often used to power frauds, scams, and political disinformation. We propose a novel forensic machine learning technique for the detection of deepfake video impersonations that leverages unnatural patterns in facial biometrics. We evaluate this technique across a large dataset of deepfake techniques and impersonations, as well as assess its reliability to video laundering and its generalization to previously unseen video deepfake generators.

Key findings

The proposed method achieves high accuracy (above 90%) in detecting face-swap and lip-sync deepfakes, demonstrating robustness to video resolution changes. The model generalizes well across different deepfake generators, although performance slightly decreases when dealing with unseen generators, but this is largely mitigated by retraining with representative samples from the new generator.

Approach

The approach extracts facial biometric vectors from video frames using ArcFace. It then calculates pairwise cosine similarities between these vectors and analyzes the distribution's statistical moments (mean, variance, etc.). These features are used to train an XGBoost classifier to distinguish between real and deepfake videos.

Datasets

DeepSpeak dataset (v1.0), Celeb-DF-v2

Model(s)

ArcFace (for feature extraction), XGBoost (for classification)

Author countries

USA

← Previous