Deepfake Network Architecture Attribution

View on arXiv ← Back to list

Authors: Tianyun Yang, Ziyao Huang, Juan Cao, Lei Li, Xirong Li

Published: 2022-02-28 14:54:30+00:00

AI Summary

This paper introduces the novel task of Deepfake Network Architecture Attribution, aiming to attribute fake images to their generator's architecture even after finetuning or retraining. A new method, DNA-Det, is proposed, leveraging pre-training on image transformations and patchwise contrastive learning to identify globally consistent architectural fingerprints.

Abstract

With the rapid progress of generation technology, it has become necessary to attribute the origin of fake images. Existing works on fake image attribution perform multi-class classification on several Generative Adversarial Network (GAN) models and obtain high accuracies. While encouraging, these works are restricted to model-level attribution, only capable of handling images generated by seen models with a specific seed, loss and dataset, which is limited in real-world scenarios when fake images may be generated by privately trained models. This motivates us to ask whether it is possible to attribute fake images to the source models' architectures even if they are finetuned or retrained under different configurations. In this work, we present the first study on Deepfake Network Architecture Attribution to attribute fake images on architecture-level. Based on an observation that GAN architecture is likely to leave globally consistent fingerprints while traces left by model weights vary in different regions, we provide a simple yet effective solution named DNA-Det for this problem. Extensive experiments on multiple cross-test setups and a large-scale dataset demonstrate the effectiveness of DNA-Det.

Key findings

DNA-Det significantly outperforms existing methods in cross-seed, cross-loss, cross-finetune, and cross-dataset settings, demonstrating its robustness and effectiveness in architecture-level attribution. The method achieves high accuracy even with diverse GANs and various image manipulations.

Approach

DNA-Det addresses the problem by using a two-step training process. First, it pre-trains on image transformation classification to focus on architecture-related features. Second, it performs GAN architecture classification using patchwise contrastive learning to strengthen the global consistency of extracted features.

Datasets

CelebA, LSUN-bedroom, and a large-scale dataset containing 59 GAN models from 10 architectures with 3 resolutions (including ProGAN, MMDGAN, SNGAN, InfoMaxGAN, CycleGAN, StackGAN2, StyleGAN, and StyleGAN2).

Model(s)

An 8-layer CNN encoder with a classification head and a projection head, trained with patchwise contrastive learning and pre-trained on image transformations.

Author countries

China

← Previous