On the Exploitation of Deepfake Model Recognition

View on arXiv ← Back to list

Authors: Luca Guarnera, Oliver Giudice, Matthias Niessner, Sebastiano Battiato

Published: 2022-04-09 16:48:23+00:00

AI Summary

This research introduces a robust pipeline for Deepfake model recognition, focusing on identifying the specific StyleGAN2 model that generated an image. By analyzing the latent space of 50 slightly different StyleGAN2 models, they trained an encoder achieving over 96% accuracy in classifying images from known models and over 94% accuracy on unseen models.

Abstract

Despite recent advances in Generative Adversarial Networks (GANs), with special focus to the Deepfake phenomenon there is no a clear understanding neither in terms of explainability nor of recognition of the involved models. In particular, the recognition of a specific GAN model that generated the deepfake image compared to many other possible models created by the same generative architecture (e.g. StyleGAN) is a task not yet completely addressed in the state-of-the-art. In this work, a robust processing pipeline to evaluate the possibility to point-out analytic fingerprints for Deepfake model recognition is presented. After exploiting the latent space of 50 slightly different models through an in-depth analysis on the generated images, a proper encoder was trained to discriminate among these models obtaining a classification accuracy of over 96%. Once demonstrated the possibility to discriminate extremely similar images, a dedicated metric exploiting the insights discovered in the latent space was introduced. By achieving a final accuracy of more than 94% for the Model Recognition task on images generated by models not employed in the training phase, this study takes an important step in countering the Deepfake phenomenon introducing a sort of signature in some sense similar to those employed in the multimedia forensics field (e.g. for camera source identification task, image ballistics task, etc).

Key findings

The proposed method achieved over 96% accuracy in classifying images from known StyleGAN2 models and over 94% accuracy on unseen models. The approach demonstrated robustness against various image manipulations, maintaining accuracy above 80% in noisy conditions. The results suggest the feasibility of Deepfake model recognition for image provenance attribution.

Approach

The approach uses a Resnet-18 encoder trained to classify images generated by 50 slightly different StyleGAN2 models. This encoder is then used to extract features, and a metric learning approach is applied to compare images and determine if they originate from the same model. The first layer of the encoder provides the most robust feature extraction.

Datasets

A dataset of 100,000 images generated by 100 slightly different StyleGAN2-ADA models, with variations in hyperparameters k-img and p. 50 models were used for training and 50 for testing.

Model(s)

Resnet-18 encoder, StyleGAN2-ADA (for generating deepfake images)

Author countries

Italy, Italy, Germany

← Previous