Wavelet-Packets for Deepfake Image Analysis and Detection

View on arXiv ← Back to list

Authors: Moritz Wolter, Felix Blanke, Raoul Heese, Jochen Garcke

Published: 2021-06-17 10:41:44+00:00

AI Summary

This paper proposes a novel deepfake image detection method using wavelet-packet transforms, offering a multi-scale spatio-frequency representation. The approach leverages differences in wavelet coefficients between real and fake images for classification, achieving competitive or improved performance compared to existing methods with smaller network sizes.

Abstract

As neural networks become able to generate realistic artificial images, they have the potential to improve movies, music, video games and make the internet an even more creative and inspiring place. Yet, the latest technology potentially enables new digital ways to lie. In response, the need for a diverse and reliable method toolbox arises to identify artificial images and other content. Previous work primarily relies on pixel-space CNNs or the Fourier transform. To the best of our knowledge, synthesized fake image analysis and detection methods based on a multi-scale wavelet representation, localized in both space and frequency, have been absent thus far. The wavelet transform conserves spatial information to a degree, which allows us to present a new analysis. Comparing the wavelet coefficients of real and fake images allows interpretation. Significant differences are identified. Additionally, this paper proposes to learn a model for the detection of synthetic images based on the wavelet-packet representation of natural and GAN-generated images. Our lightweight forensic classifiers exhibit competitive or improved performance at comparatively small network sizes, as we demonstrate on the FFHQ, CelebA and LSUN source identification problems. Furthermore, we study the binary FaceForensics++ fake-detection problem.

Key findings

The wavelet-packet approach demonstrates competitive or improved performance compared to pixel-space CNNs and Fourier transform-based methods, particularly with smaller network sizes. The method shows robustness to reduced training data. Fusion networks combining wavelet-packet and Fourier features achieve state-of-the-art results on some datasets.

Approach

The authors use wavelet-packet transforms to represent images in a multi-scale spatio-frequency domain. They identify significant differences in wavelet coefficients between real and fake images. These differences are then used to train lightweight forensic classifiers for deepfake detection.

Datasets

FFHQ, CelebA, LSUN, FaceForensics++

Model(s)

Linear regression models, CNNs (various architectures adapted for wavelet-packet inputs), fusion networks combining wavelet-packet and Fourier features.

Author countries

Germany

← Previous