UnGANable: Defending Against GAN-based Face Manipulation

View on arXiv ← Back to list

Authors: Zheng Li, Ning Yu, Ahmed Salem, Michael Backes, Mario Fritz, Yang Zhang

Published: 2022-10-03 14:20:01+00:00

AI Summary

UnGANable is the first defense system against GAN-inversion-based face manipulation. It achieves this by searching for 'cloaked images' visually similar to the originals but hindering the GAN inversion process, thereby preventing malicious face manipulation.

Abstract

Deepfakes pose severe threats of visual misinformation to our society. One representative deepfake application is face manipulation that modifies a victim's facial attributes in an image, e.g., changing her age or hair color. The state-of-the-art face manipulation techniques rely on Generative Adversarial Networks (GANs). In this paper, we propose the first defense system, namely UnGANable, against GAN-inversion-based face manipulation. In specific, UnGANable focuses on defending GAN inversion, an essential step for face manipulation. Its core technique is to search for alternative images (called cloaked images) around the original images (called target images) in image space. When posted online, these cloaked images can jeopardize the GAN inversion process. We consider two state-of-the-art inversion techniques including optimization-based inversion and hybrid inversion, and design five different defenses under five scenarios depending on the defender's background knowledge. Extensive experiments on four popular GAN models trained on two benchmark face datasets show that UnGANable achieves remarkable effectiveness and utility performance, and outperforms multiple baseline methods. We further investigate four adaptive adversaries to bypass UnGANable and show that some of them are slightly effective.

Key findings

UnGANable demonstrates remarkable effectiveness and utility, outperforming multiple baseline methods. While some adaptive adversaries were slightly effective at bypassing the defense, the overall results highlight the potential of this approach.

Approach

UnGANable defends against GAN inversion by generating 'cloaked images' – visually similar alternatives to the original images – that disrupt the inversion process. It designs five different cloaking methods based on varying levels of defender knowledge and targets two state-of-the-art inversion techniques.

Datasets

Two benchmark face datasets (names not specified in the provided abstract)

Model(s)

Four popular GAN models (names not specified in the provided abstract)

Author countries

Germany, USA

← Previous