MRI-GAN: A Generalized Approach to Detect DeepFakes using Perceptual Image Assessment

View on arXiv ← Back to list

Authors: Pratikkumar Prajapati, Chris Pollett

Published: 2022-02-28 21:53:07+00:00

AI Summary

This paper proposes MRI-GAN, a Generative Adversarial Network (GAN)-based framework for deepfake detection that leverages perceptual differences in images using Structural Similarity Index Measurement (SSIM). A plain frames-based model achieved 91% accuracy, while the MRI-GAN model, using SSIM, achieved 74% accuracy, with potential for improvement through hyperparameter tuning and loss function modifications.

Abstract

DeepFakes are synthetic videos generated by swapping a face of an original image with the face of somebody else. In this paper, we describe our work to develop general, deep learning-based models to classify DeepFake content. We propose a novel framework for using Generative Adversarial Network (GAN)-based models, we call MRI-GAN, that utilizes perceptual differences in images to detect synthesized videos. We test our MRI-GAN approach and a plain-frames-based model using the DeepFake Detection Challenge Dataset. Our plain frames-based-model achieves 91% test accuracy and a model which uses our MRI-GAN framework with Structural Similarity Index Measurement (SSIM) for the perceptual differences achieves 74% test accuracy. The results of MRI-GAN are preliminary and may be improved further by modifying the choice of loss function, tuning hyper-parameters, or by using a more advanced perceptual similarity metric.

Key findings

A plain frames-based model achieved 91% test accuracy, while the MRI-GAN model achieved 74% accuracy. The MRI-GAN results are preliminary and can be improved. The authors acknowledge that their models haven't been tested on the private DFDC test set.

Approach

MRI-GAN generates an image representing the perceptual difference (using SSIM) between a given face and a corresponding real face. A blank image indicates a real face, while artifacts indicate a deepfake. This 'MRI' image, along with other features, is used by a CNN to classify the video.

Datasets

DeepFake Detection Challenge Dataset (DFDC), Celeb-DF-v2, FDF, FFHQ

Model(s)

Generative Adversarial Network (GAN) - MRI-GAN, EfficientNet B0, Multi-Layer Perceptron (MLP), Multi-task Cascaded Convolutional Networks (MTCNN)

Author countries

USA

← Previous