Identity-Driven DeepFake Detection

View on arXiv ← Back to list

Authors: Xiaoyi Dong, Jianmin Bao, Dongdong Chen, Weiming Zhang, Nenghai Yu, Dong Chen, Fang Wen, Baining Guo

Published: 2020-12-07 18:59:08+00:00

AI Summary

This paper introduces Identity-Driven DeepFake Detection, an approach that uses reference images of a target identity to determine if a suspect image/video contains the same identity, unlike artifact-based methods. It introduces a new large-scale dataset, Vox-DeepFake, and a baseline algorithm, OuterFace, which achieves superior accuracy even without training on fake videos.

Abstract

DeepFake detection has so far been dominated by ``artifact-driven'' methods and the detection performance significantly degrades when either the type of image artifacts is unknown or the artifacts are simply too hard to find. In this work, we present an alternative approach: Identity-Driven DeepFake Detection. Our approach takes as input the suspect image/video as well as the target identity information (a reference image or video). We output a decision on whether the identity in the suspect image/video is the same as the target identity. Our motivation is to prevent the most common and harmful DeepFakes that spread false information of a targeted person. The identity-based approach is fundamentally different in that it does not attempt to detect image artifacts. Instead, it focuses on whether the identity in the suspect image/video is true. To facilitate research on identity-based detection, we present a new large scale dataset ``Vox-DeepFake, in which each suspect content is associated with multiple reference images collected from videos of a target identity. We also present a simple identity-based detection algorithm called the OuterFace, which may serve as a baseline for further research. Even trained without fake videos, the OuterFace algorithm achieves superior detection accuracy and generalizes well to different DeepFake methods, and is robust with respect to video degradation techniques -- a performance not achievable with existing detection algorithms.

Key findings

OuterFace, trained only on real face images from VggFace2, outperforms existing artifact-driven methods on various datasets and is robust to video degradation techniques. The method's performance improves with the addition of multiple reference images and is less sensitive to variations in reference pose.

Approach

The proposed approach uses a reference image along with a suspect image/video to verify if both contain the same identity, framing deepfake detection as an identity verification problem rather than an artifact detection problem. The OuterFace algorithm masks inner face regions to force the model to learn robust identity representations from the outer face.

Datasets

Vox-DeepFake, FaceForensics++, Google DeepFake Detection, Celeb-DeepFake, VggFace2, LFW (for supplementary experiments)

Model(s)

MobileNet with a cosine-based softmax loss (ArcFace) for identity embedding. Face landmark detection model used for preprocessing.

Author countries

China, USA

← Previous