Deepfake Detection without Deepfakes: Generalization via Synthetic Frequency Patterns Injection

View on arXiv ← Back to list

Authors: Davide Alessandro Coccomini, Roberto Caldelli, Claudio Gennaro, Giuseppe Fiameni, Giuseppe Amato, Fabrizio Falchi

Published: 2024-03-20 10:33:10+00:00

AI Summary

This paper proposes a novel deepfake detection method that enhances generalization by training models on pristine images with injected synthetic frequency patterns. These patterns simulate deepfake artifacts without being tied to specific generation techniques, leading to state-of-the-art detection performance across diverse generation methods.

Abstract

Deepfake detectors are typically trained on large sets of pristine and generated images, resulting in limited generalization capacity; they excel at identifying deepfakes created through methods encountered during training but struggle with those generated by unknown techniques. This paper introduces a learning approach aimed at significantly enhancing the generalization capabilities of deepfake detectors. Our method takes inspiration from the unique fingerprints that image generation processes consistently introduce into the frequency domain. These fingerprints manifest as structured and distinctly recognizable frequency patterns. We propose to train detectors using only pristine images injecting in part of them crafted frequency patterns, simulating the effects of various deepfake generation techniques without being specific to any. These synthetic patterns are based on generic shapes, grids, or auras. We evaluated our approach using diverse architectures across 25 different generation methods. The models trained with our approach were able to perform state-of-the-art deepfake detection, demonstrating also superior generalization capabilities in comparison with previous methods. Indeed, they are untied to any specific generation technique and can effectively identify deepfakes regardless of how they were made.

Key findings

Models trained with the proposed method achieved state-of-the-art deepfake detection performance, significantly outperforming methods trained on actual deepfakes. The superior generalization capability allows effective identification of deepfakes regardless of the generation technique. Training for only one epoch was crucial to avoid overfitting to the synthetic patterns.

Approach

The approach injects synthetic frequency patterns, inspired by common deepfake artifacts, into pristine images. Models are trained to distinguish between pristine and pattern-injected images, learning to identify general deepfake artifacts rather than specific generator fingerprints. This method uses only pristine images during training.

Datasets

MSCOCO (for pristine images), datasets from [16] and [2] (for fake images generated by 25 different methods, including StyleGAN, StyleGAN2, ProGAN, RelGAN, BETA_B, BigGAN, BicycleGAN, DAGAN_C, DFCVAE, DRIT, GANANIME, LOGAN, PIX2PIX, SAGAN, SRRNET, RSGAN-H, UNIT, ADM, ADM-G, DDIM, DDPM, GLIDE, LDM, Stable Diffusion, and Stable Diffusion XL), and 5,000 images generated using Stable Diffusion XL from MSCOCO validation set captions.

Model(s)

ResNet50 and Swin-Base (both pretrained on ImageNet).

Author countries

Italy

← Previous