Herd Mentality in Augmentation -- Not a Good Idea! A Robust Multi-stage Approach towards Deepfake Detection

View on arXiv ← Back to list

Authors: Monu, Rohan Raju Dhanakshirur

Published: 2024-10-07 19:51:46+00:00

AI Summary

This paper presents a multi-stage approach for improving deepfake detection accuracy. The approach addresses issues in existing models, such as inappropriate data augmentation and over-reliance on specific facial features, by incorporating weighted loss, modified augmentation techniques (rotation and flipping), and masked eye pre-training. This results in significant improvements in F1 score and accuracy on the Celeb-DF v2 dataset.

Abstract

The rapid increase in deepfake technology has raised significant concerns about digital media integrity. Detecting deepfakes is crucial for safeguarding digital media. However, most standard image classifiers fail to distinguish between fake and real faces. Our analysis reveals that this failure is due to the model's inability to explicitly focus on the artefacts typically in deepfakes. We propose an enhanced architecture based on the GenConViT model, which incorporates weighted loss and update augmentation techniques and includes masked eye pretraining. This proposed model improves the F1 score by 1.71% and the accuracy by 4.34% on the Celeb-DF v2 dataset. The source code for our model is available at https://github.com/Monu-Khicher-1/multi-stage-learning

Key findings

The proposed multi-stage approach improves the F1 score by 1.71% and accuracy by 4.34% on the Celeb-DF v2 dataset compared to the original GenConViT model. Ablation studies demonstrate the effectiveness of each component of the proposed method. The improved model achieves an F1 score of 95.21% and accuracy of 98.36%.

Approach

The authors enhance the GenConViT model by employing a weighted loss function to address class imbalance, replacing ineffective augmentations with simple rotation and flipping, and using masked eye pre-training to force the model to learn from features beyond the eyes. These modifications are implemented in a multi-stage training process.

Datasets

Celeb-DF v2

Model(s)

GenConViT (modified)

Author countries

India

← Previous