Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Learning

View on arXiv ← Back to list

Authors: Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, Yunchao Wei

Published: 2024-03-12 01:28:00+00:00

AI Summary

This paper proposes FreqNet, a novel frequency-aware deepfake detector that improves generalizability by focusing on high-frequency information in both spatial and channel dimensions of images and features. It incorporates a frequency domain learning module using convolutional layers on the phase and amplitude spectrums, achieving state-of-the-art performance with fewer parameters.

Abstract

This research addresses the challenge of developing a universal deepfake detector that can effectively identify unseen deepfake images despite limited training data. Existing frequency-based paradigms have relied on frequency-level artifacts introduced during the up-sampling in GAN pipelines to detect forgeries. However, the rapid advancements in synthesis technology have led to specific artifacts for each generation model. Consequently, these detectors have exhibited a lack of proficiency in learning the frequency domain and tend to overfit to the artifacts present in the training data, leading to suboptimal performance on unseen sources. To address this issue, we introduce a novel frequency-aware approach called FreqNet, centered around frequency domain learning, specifically designed to enhance the generalizability of deepfake detectors. Our method forces the detector to continuously focus on high-frequency information, exploiting high-frequency representation of features across spatial and channel dimensions. Additionally, we incorporate a straightforward frequency domain learning module to learn source-agnostic features. It involves convolutional layers applied to both the phase spectrum and amplitude spectrum between the Fast Fourier Transform (FFT) and Inverse Fast Fourier Transform (iFFT). Extensive experimentation involving 17 GANs demonstrates the effectiveness of our proposed method, showcasing state-of-the-art performance (+9.8%) while requiring fewer parameters. The code is available at {cred url{https://github.com/chuangchuangtan/FreqNet-DeepfakeDetection}}.

Key findings

FreqNet achieves state-of-the-art performance (+9.8% improvement in mean accuracy) compared to existing models while using significantly fewer parameters (1.9 million vs. 304 million). The results demonstrate superior generalizability across 17 different GAN models and various image categories. Ablation studies confirm the effectiveness of the proposed modules.

Approach

FreqNet uses FFT to transform images and features to the frequency domain. It then extracts high-frequency components and applies convolutional layers to the phase and amplitude spectrums before inverse transformation. This process enhances the detector's focus on high-frequency information and reduces overfitting to specific artifacts.

Datasets

ForenSynths (training and test sets), LSUN, ImageNet, CelebA, CelebA-HQ, COCO, FaceForensics++, and images generated by 17 different GANs (ProGAN, StyleGAN, StyleGAN2, BigGAN, CycleGAN, StarGAN, GauGAN, Deepfake, AttGAN, BEGAN, CramerGAN, InfoMaxGAN, MMDGAN, RelGAN, S3GAN, SNGAN, STGAN)

Model(s)

A lightweight CNN classifier with residual convolutional blocks and a frequency domain learning module (FreqNet)

Author countries

China, Singapore

← Previous