DiffFake: Exposing Deepfakes using Differential Anomaly Detection

View on arXiv ← Back to list

Authors: Sotirios Stamnas, Victor Sanchez

Published: 2025-02-22 14:50:53+00:00

AI Summary

DiffFake is a novel deepfake detector that uses a differential anomaly detection framework. Instead of binary classification, it learns natural facial changes between image pairs to identify unnatural changes indicative of deepfakes, improving generalization to unseen deepfake generation techniques.

Abstract

Traditional deepfake detectors have dealt with the detection problem as a binary classification task. This approach can achieve satisfactory results in cases where samples of a given deepfake generation technique have been seen during training, but can easily fail with deepfakes generated by other techniques. In this paper, we propose DiffFake, a novel deepfake detector that approaches the detection problem as an anomaly detection task. Specifically, DiffFake learns natural changes that occur between two facial images of the same person by leveraging a differential anomaly detection framework. This is done by combining pairs of deep face embeddings and using them to train an anomaly detection model. We further propose to train a feature extractor on pseudo-deepfakes with global and local artifacts, to extract meaningful and generalizable features that can then be used to train the anomaly detection model. We perform extensive experiments on five different deepfake datasets and show that our method can match and sometimes even exceed the performance of state-of-the-art competitors.

Key findings

DiffFake achieves competitive or superior performance compared to state-of-the-art methods across various deepfake datasets and experimental settings (cross-manipulation, cross-dataset, degrading video quality). The differential anomaly detection framework significantly improves generalization, particularly when dealing with unseen deepfake generation techniques. The choice of feature combination and anomaly detection model impacts performance.

Approach

DiffFake trains a feature extractor on real images and pseudo-deepfakes (with both local and global artifacts) to extract embeddings from image pairs. An anomaly detection model (GMM) trained only on real image pairs then classifies whether the input pair exhibits natural or deepfake-like changes.

Datasets

FF++, CDF, DF1.0, FSh, FNet

Model(s)

EfficientNet-b4 (backbone), Gaussian Mixture Model (GMM) (anomaly detection model)

Author countries

United Kingdom

← Previous