Improving Fairness in Deepfake Detection

View on arXiv ← Back to list

Authors: Yan Ju, Shu Hu, Shan Jia, George H. Chen, Siwei Lyu

Published: 2023-06-29 02:19:49+00:00

AI Summary

This paper proposes two novel loss functions for improving fairness in deepfake detection. These functions address data imbalances, handling scenarios with and without demographic information, and can be applied to existing deepfake detectors to enhance fairness without significant performance loss.

Abstract

Despite the development of effective deepfake detectors in recent years, recent studies have demonstrated that biases in the data used to train these detectors can lead to disparities in detection accuracy across different races and genders. This can result in different groups being unfairly targeted or excluded from detection, allowing undetected deepfakes to manipulate public opinion and erode trust in a deepfake detection model. While existing studies have focused on evaluating fairness of deepfake detectors, to the best of our knowledge, no method has been developed to encourage fairness in deepfake detection at the algorithm level. In this work, we make the first attempt to improve deepfake detection fairness by proposing novel loss functions that handle both the setting where demographic information (eg, annotations of race and gender) is available as well as the case where this information is absent. Fundamentally, both approaches can be used to convert many existing deepfake detectors into ones that encourages fairness. Extensive experiments on four deepfake datasets and five deepfake detectors demonstrate the effectiveness and flexibility of our approach in improving deepfake detection fairness. Our code is available at https://github.com/littlejuyan/DF_Fairness.

Key findings

Extensive experiments demonstrate the effectiveness of both DAG-FDD and DAW-FDD in improving fairness across gender, race, and intersectional groups on multiple datasets and deepfake detectors. The methods achieve this fairness improvement without substantial decreases in overall deepfake detection accuracy; in some cases, even improving accuracy.

Approach

The authors propose two methods: DAG-FDD, which is demographic-agnostic and uses a Conditional Value-at-Risk (CVaR) loss function, and DAW-FDD, which is demographic-aware and uses a hierarchical CVaR approach to handle imbalances in both demographic groups and real/fake examples. Both methods are designed to modify existing deepfake detectors.

Datasets

FaceForensics++ (FF++), Celeb-DF, DeepFakeDetection (DFD), Deepfake Detection Challenge (DFDC), DF-Platter

Model(s)

Xception, ResNet-50, EfficientNet-B3, DSP-FWA, RECCE

Author countries

USA, USA, USA, USA, USA

← Previous