Detecting GAN-generated Imagery using Color Cues

Authors: Scott McCloskey, Michael Albright

Published: 2018-12-19 21:12:00+00:00

AI Summary

This paper addresses the growing problem of detecting GAN-generated imagery, or deepfakes, which contribute to online disinformation. The authors analyze a popular GAN implementation's generator network and identify two distinct ways its color treatment differs from real cameras. These color cues, specifically related to normalization and color formation, are then leveraged to effectively distinguish synthetic GAN imagery from real camera images.

Abstract

Image forensics is an increasingly relevant problem, as it can potentially address online disinformation campaigns and mitigate problematic aspects of social media. Of particular interest, given its recent successes, is the detection of imagery produced by Generative Adversarial Networks (GANs), e.g. `deepfakes'. Leveraging large training sets and extensive computing resources, recent work has shown that GANs can be trained to generate synthetic imagery which is (in some ways) indistinguishable from real imagery. We analyze the structure of the generating network of a popular GAN implementation, and show that the network's treatment of color is markedly different from a real camera in two ways. We further show that these two cues can be used to distinguish GAN-generated imagery from camera imagery, demonstrating effective discrimination between GAN imagery and real camera images used to train the GAN.

Key findings

The saturation-based forensic, which measures the frequency of over-exposed pixels, showed good discrimination between GAN-generated and camera imagery, achieving a 0.7 AUC for fully GAN-generated images and 0.61 for images with GAN-generated faces. The color image forensic, based on color statistics, performed poorly, yielding AUCs of 0.56 and 0.54, which was only slightly better than random and possibly limited by training data and evaluation set characteristics.

Approach

The authors propose two detection methods: one based on color image forensics and another on saturation statistics. The color forensic analyzes pixel-wise correlation in rg chromaticity space, using a pre-trained INH network. The saturation-based forensic measures the frequency of saturated and under-exposed pixels, classified by a linear Support Vector Machine (SVM).

Datasets

Media Forensics Challenge 2018 (GAN Crop images, GAN Full images), LSUN, ImageNet.

Model(s)

INH network (similar to VGG), Linear Support Vector Machine (SVM).

Author countries

USA

← Previous