Face X-ray for More General Face Forgery Detection

View on arXiv ← Back to list

Authors: Lingzhi Li, Jianmin Bao, Ting Zhang, Hao Yang, Dong Chen, Fang Wen, Baining Guo

Published: 2019-12-31 17:57:56+00:00

AI Summary

This paper introduces "face X-ray," a novel image representation for detecting face forgeries. Face X-ray reveals blending boundaries in forged images and is trained only on real images, making it robust to unseen forgery techniques.

Abstract

In this paper we propose a novel image representation called face X-ray for detecting forgery in face images. The face X-ray of an input face image is a greyscale image that reveals whether the input image can be decomposed into the blending of two images from different sources. It does so by showing the blending boundary for a forged image and the absence of blending for a real image. We observe that most existing face manipulation methods share a common step: blending the altered face into an existing background image. For this reason, face X-ray provides an effective way for detecting forgery generated by most existing face manipulation algorithms. Face X-ray is general in the sense that it only assumes the existence of a blending step and does not rely on any knowledge of the artifacts associated with a specific face manipulation technique. Indeed, the algorithm for computing face X-ray can be trained without fake images generated by any of the state-of-the-art face manipulation methods. Extensive experiments show that face X-ray remains effective when applied to forgery generated by unseen face manipulation techniques, while most existing face forgery detection or deepfake detection algorithms experience a significant performance drop.

Key findings

Face X-ray demonstrates high accuracy in detecting unseen forgery techniques, outperforming existing methods. The model's performance is robust even when trained solely on blended real images, showcasing its generalizability. It also accurately predicts face X-rays, providing visual explanations of its decisions.

Approach

The approach detects forgeries by identifying blending boundaries in images. It generates a grayscale "face X-ray" image that highlights these boundaries, indicating manipulation. This is achieved using a convolutional neural network trained on real images blended together to simulate manipulation.

Datasets

FaceForensics++, a large-scale video dataset with four types of facial manipulations (DeepFakes, Face2Face, FaceSwap, NeuralTextures); a dataset of blended images created from real images in FaceForensics++; DeepfakeDetection, Deepfake Detection Challenge, Celeb-DF.

Model(s)

HRNet (a high-resolution neural network architecture)

Author countries

China,USA

← Previous