A Dataless FaceSwap Detection Approach Using Synthetic Images

View on arXiv ← Back to list

Authors: Anubhav Jain, Nasir Memon, Julian Togelius

Published: 2022-12-05 19:49:45+00:00

AI Summary

This paper proposes a deepfake detection method that utilizes synthetic data generated by StyleGAN3, eliminating the need for real data and reducing biases present in real-world datasets. The approach achieves comparable performance to methods using real data and demonstrates better generalization capabilities when fine-tuned with a small amount of real data.

Abstract

Face swapping technology used to create Deepfakes has advanced significantly over the past few years and now enables us to create realistic facial manipulations. Current deep learning algorithms to detect deepfakes have shown promising results, however, they require large amounts of training data, and as we show they are biased towards a particular ethnicity. We propose a deepfake detection methodology that eliminates the need for any real data by making use of synthetically generated data using StyleGAN3. This not only performs at par with the traditional training methodology of using real data but it shows better generalization capabilities when finetuned with a small amount of real data. Furthermore, this also reduces biases created by facial image datasets that might have sparse data from particular ethnicities.

Key findings

The dataless approach using synthetic data achieves comparable performance to methods trained on real data. Fine-tuning with a small amount of real data further improves performance and generalization. The synthetic data approach also mitigates biases observed in models trained on real data, leading to fairer and less biased detection.

Approach

The authors generate synthetic facial images using StyleGAN3 and then create face swaps using these images with existing face-swapping models (SimSwap and SberSwap). An Xception network is trained to distinguish between the synthetic originals and the face swaps, and the model is fine-tuned with a small amount of real data to improve generalization.

Datasets

Flickr Faces High Quality (FFHQ), CelebA-HQ, Amsterdam Dynamic Facial Expression Set (ADFES)

Model(s)

StyleGAN3 (for data generation), SimSwap and SberSwap (for face swapping), Xception (for deepfake detection)

Author countries

USA

← Previous