Data-Independent Operator: A Training-Free Artifact Representation Extractor for Generalizable Deepfake Detection

View on arXiv ← Back to list

Authors: Chuangchuang Tan, Ping Liu, RenShuai Tao, Huan Liu, Yao Zhao, Baoyuan Wu, Yunchao Wei

Published: 2024-03-11 15:22:28+00:00

AI Summary

This paper introduces the Data-Independent Operator (DIO) framework for generalizable deepfake detection, using fixed, training-free filters to extract artifact representations instead of large, trained models. DIO achieves state-of-the-art performance by focusing on generalizable artifact clues, improving generalization to unseen sources.

Abstract

Recently, the proliferation of increasingly realistic synthetic images generated by various generative adversarial networks has increased the risk of misuse. Consequently, there is a pressing need to develop a generalizable detector for accurately recognizing fake images. The conventional methods rely on generating diverse training sources or large pretrained models. In this work, we show that, on the contrary, the small and training-free filter is sufficient to capture more general artifact representations. Due to its unbias towards both the training and test sources, we define it as Data-Independent Operator (DIO) to achieve appealing improvements on unseen sources. In our framework, handcrafted filters and the randomly-initialized convolutional layer can be used as the training-free artifact representations extractor with excellent results. With the data-independent operator of a popular classifier, such as Resnet50, one could already reach a new state-of-the-art without bells and whistles. We evaluate the effectiveness of the DIO on 33 generation models, even DALLE and Midjourney. Our detector achieves a remarkable improvement of $13.3%$, establishing a new state-of-the-art performance. The DIO and its extension can serve as strong baselines for future methods. The code is available at url{https://github.com/chuangchuangtan/Data-Independent-Operator}.

Key findings

The DIO framework significantly improves generalization performance compared to existing methods, achieving a 13.3% improvement in state-of-the-art performance across 33 generation models. The use of randomly initialized convolutional layers shows that prior knowledge isn't essential for effective artifact representation extraction. MDIO, combining multiple DIOs, further enhances performance.

Approach

The proposed method uses fixed filters (handcrafted or randomly initialized convolutional layers) as a 'Data-Independent Operator' (DIO) to extract artifact representations from images. These representations, independent of training data, are then fed into a pre-trained classifier (like ResNet50) for deepfake detection. Multiple DIOs (MDIO) can be cascaded for improved performance.

Datasets

ForenSynths, GANGen-Detection, DiffusionForensics, UniversalFakeDetect, AIGCDetectBenchmark; These datasets include images generated by 33 different GAN and diffusion models.

Model(s)

ResNet50 (primarily), Xception, ResNet50 with layers removed, DIOs employ handcrafted filters (Laplacian of Gaussian, Laplacian, Avgpool, Sobel), pre-trained convolutional layers (from VGG16, ResNet101, InceptionV3, CLIP-ResNet50, DeeplabV3) and randomly initialized convolutional layers.

Author countries

China, USA

← Previous