FairForensics: Seeing Expressions and Parsing Demographics via Vision-Language Modeling for Generalizable Fair Deepfake Detection

Authors: Yaning Zhang, Jiao Wu, Zan Gao, Linlin Shen

Published: 2026-08-03 03:51:14+00:00

AI Summary

This paper introduces FairForensics, a novel expression and demographic perceptual vision-language model for generalizable fair deepfake detection (FDD). FairForensics enhances face forgery generalization by using an expression encoder and an expression-perceptual visual encoder that integrates global appearance and expression forgery features while mitigating identity bias. It also incorporates demographic-aware fairness regularization through a demographic-guided language encoder and a population-aware prototype fairness objective, all evaluated on a newly constructed demographically balanced FDD benchmark.

Abstract

The challenge of fair deepfake detection (FDD) has attracted increasing attention. Existing fairness-enhanced detectors often suffer from suboptimal generalization to unseen manipulations and fairness across demographic groups. They are typically developed and evaluated on demographically imbalanced distributions, resulting in biased predictions toward minority groups. In this paper, we construct a novel demographically balanced FDD benchmark to train and evaluate the fairness of detectors under both balanced and imbalanced population scenarios. Additionally, we introduce a novel expression and demographic perceptual vision-language model, termed FairForensics, for generalizable fair deepfake detection. FairForensics conducts face forgery generalization enhancement and demographic-aware fairness regularization. During face forgery generalization enhancement, built upon the novel observation of significant distribution differences between pristine and forged expression vectors, we design an expression encoder to capture high-level expression-guided forgery patterns, and an expression-perceptual visual encoder that integrates global appearance and expression forgery features while mitigating identity bias using an identity-aware patch perturbation module. Under demographic-aware fairness regularization, we propose a demographic-guided language encoder to extract population-aware global language embeddings, which boosts the decoupling of forgery features from demographic information via vision-language alignment. We devise a population-aware prototype fairness objective to enforce both inter-class separability and intra-class alignment across demographic subgroups. Extensive experiments on our balanced demographic benchmark show that our method achieves the state-of-the-art in terms of generalization and fairness.

Key findings

The FairForensics model achieves state-of-the-art performance in both generalization and fairness on a demographically balanced benchmark. Training on balanced demographic data significantly reduces fairness disparities and improves cross-domain generalization. The proposed modules, including the expression encoder, expression-perceptual visual encoder with identity-aware patch perturbation, demographic-guided language encoder, and population-aware prototype fairness objective, are all crucial and complementary for achieving strong detection and fairness performance.

Approach

FairForensics tackles fair deepfake detection by enhancing forgery generalization and applying demographic-aware fairness regularization. It uses an expression encoder to capture expression-guided forgery patterns and an expression-perceptual visual encoder that integrates these with global appearance features, mitigating identity bias with an identity-aware patch perturbation module. Demographic fairness is achieved through a demographic-guided language encoder for vision-language alignment and a population-aware prototype fairness objective that enforces inter-class separability and intra-class alignment across demographic subgroups.

Datasets

GenFace (modified to be demographically balanced), FF++, Celeb-DF, DFDC, DF-1.0, AffectNet8 (for POSTER pretraining), RAFDB (for SSFER pretraining), FERPLUS (for SSFER pretraining)

Model(s)

Xception, ResNet, VGG, ViT, CViT, CAEL, CLIP, MFCLIP, ForensicsAdapter (FA), DAG-FDD, DAW-FDD, PG-FDD, FairAdapter, RSEF-FDD, Ding et al., POSTER, SSFER

Author countries

China

← Previous