FrePGAN: Robust Deepfake Detection Using Frequency-level Perturbations

View on arXiv ← Back to list

Authors: Yonghyun Jeong, Doyeon Kim, Youngmin Ro, Jongwon Choi

Published: 2022-02-07 16:45:11+00:00

AI Summary

This paper introduces FrePGAN, a deepfake detection framework that addresses overfitting issues by focusing on frequency-level artifacts in generated images. FrePGAN generates perturbation maps to reduce frequency differences between real and fake images, improving generalization across various GAN models and training settings.

Abstract

Various deepfake detectors have been proposed, but challenges still exist to detect images of unknown categories or GAN models outside of the training settings. Such issues arise from the overfitting issue, which we discover from our own analysis and the previous studies to originate from the frequency-level artifacts in generated images. We find that ignoring the frequency-level artifacts can improve the detector's generalization across various GAN models, but it can reduce the model's performance for the trained GAN models. Thus, we design a framework to generalize the deepfake detector for both the known and unseen GAN models. Our framework generates the frequency-level perturbation maps to make the generated images indistinguishable from the real images. By updating the deepfake detector along with the training of the perturbation generator, our model is trained to detect the frequency-level artifacts at the initial iterations and consider the image-level irregularities at the last iterations. For experiments, we design new test scenarios varying from the training settings in GAN models, color manipulations, and object categories. Numerous experiments validate the state-of-the-art performance of our deepfake detector.

Key findings

FrePGAN achieves state-of-the-art performance in deepfake detection across various test scenarios, including unknown categories, GAN models, image manipulations, and resolutions. The alternating training of the perturbation generator and classifier enhances generalization, and the generated perturbations effectively reduce the influence of frequency-level artifacts.

Approach

FrePGAN uses a two-module framework: a Frequency Perturbation GAN (FrePGAN) that generates perturbation maps to mitigate frequency-level artifacts in deepfake images, and a deepfake classifier that distinguishes between real and perturbed images. The two modules are trained alternately, focusing initially on frequency artifacts and later on general image irregularities.

Datasets

FFHQ, LSUN, Imagenet, CelebA, COCO, Deepfake dataset (a combination of various videos collected online with partially generated images reconstructed by face-swapping models), and ProGAN dataset (for various experiments with different objects, resolutions, and manipulations).

Model(s)

VGG model (for perturbation map generator and discriminator), ResNet-50 (pre-trained on ImageNet for the deepfake classifier), and a custom GAN architecture (FrePGAN).

Author countries

Korea

← Previous