FakeLocator: Robust Localization of GAN-Based Face Manipulations

Authors: Yihao Huang, Felix Juefei-Xu, Qing Guo, Yang Liu, Geguang Pu

Published: 2020-01-27 06:15:01+00:00

Comment: 16 pages, accepted to IEEE Transactions on Information Forensics and Security

AI Summary

This paper introduces FakeLocator, a novel approach for robustly localizing GAN-based face manipulations by leveraging imperfections in GAN upsampling methods. FakeLocator generates high-resolution gray-scale fakeness maps, improving localization accuracy through an attention mechanism for cross-attribute universality and partial data augmentation with single sample clustering for cross-method universality. Experiments show its superior performance over baselines and strong robustness against various real-world image degradations.

Abstract

Full face synthesis and partial face manipulation by virtue of the generative adversarial networks (GANs) and its variants have raised wide public concerns. In the multi-media forensics area, detecting and ultimately locating the image forgery has become an imperative task. In this work, we investigate the architecture of existing GAN-based face manipulation methods and observe that the imperfection of upsampling methods therewithin could be served as an important asset for GAN-synthesized fake image detection and forgery localization. Based on this basic observation, we have proposed a novel approach, termed FakeLocator, to obtain high localization accuracy, at full resolution, on manipulated facial images. To the best of our knowledge, this is the very first attempt to solve the GAN-based fake localization problem with a gray-scale fakeness map that preserves more information of fake regions. To improve the universality of FakeLocator across multifarious facial attributes, we introduce an attention mechanism to guide the training of the model. To improve the universality of FakeLocator across different DeepFake methods, we propose partial data augmentation and single sample clustering on the training images. Experimental results on popular FaceForensics++, DFFD datasets and seven different state-of-the-art GAN-based face generation methods have shown the effectiveness of our method. Compared with the baselines, our method performs better on various metrics. Moreover, the proposed method is robust against various real-world facial image degradations such as JPEG compression, low-resolution, noise, and blur.

Key findings

FakeLocator achieves state-of-the-art localization accuracy at full resolution on manipulated facial images, significantly outperforming prior works across various metrics like PSNR, SSIM, and COSS. The method demonstrates strong robustness against common real-world image degradations (e.g., JPEG compression, blur, noise, low-resolution) and exhibits improved universality across diverse facial attributes and GAN-based manipulation techniques.

Approach

FakeLocator identifies unique artifacts introduced by GANs' upsampling imperfections, using an encoder-decoder network to predict full-resolution gray-scale fakeness maps. An attention mechanism, guided by face parsing, enhances cross-attribute universality by focusing on manipulated regions. Furthermore, partial data augmentation and single sample clustering are employed to improve the model's universality across different GAN-based manipulation methods.

Datasets

FaceForensics++, DFFD, CelebFaces Attributes (CelebA), Flickr-Faces-HQ (FFHQ), UADFV, Celeb-DF

Model(s)

Deeplabv3 (with ResNet-101 backbone) as the encoder-decoder architecture. A custom classification branch consisting of four convolutional layers, a max-pooling layer, three fully connected layers, and a Sigmoid layer.

Author countries

China, USA, Singapore

← Previous