Cyber Vaccine for Deepfake Immunity

View on arXiv ← Back to list

Authors: Ching-Chun Chang, Huy Hong Nguyen, Junichi Yamagishi, Isao Echizen

Published: 2023-03-05 12:29:44+00:00

AI Summary

This paper introduces a novel 'cyber vaccination' approach to deepfake restoration, aiming for attack-agnostic immunity. It uses face masking as a powerful adversarial training method to create a system with a 'vaccinator' and a 'neutraliser' to both vaccinate against and restore manipulated facial content.

Abstract

Deepfakes pose an evolving threat to cybersecurity, which calls for the development of automated countermeasures. While considerable forensic research has been devoted to the detection and localisation of deepfakes, solutions for reversing fake to real are yet to be developed. In this study, we introduce cyber vaccination for conferring immunity to deepfakes. Analogous to biological vaccination that injects antigens to induce immunity prior to infection by an actual pathogen, cyber vaccination simulates deepfakes and performs adversarial training to build a defensive immune system. Aiming at building up attack-agnostic immunity with limited computational resources, we propose to simulate various deepfakes with one single overpowered attack: face masking. The proposed immune system consists of a vaccinator for inducing immunity and a neutraliser for recovering facial content. Experimental evaluations demonstrate effective immunity to face replacement, face reenactment and various types of corruptions.

Key findings

The proposed cyber vaccination system demonstrates effective immunity to face replacement and face reenactment. The vaccinated images show high imperceptibility and the neutraliser achieves effective restoration even under various corruption conditions. A validator accurately distinguishes between vaccinated and unvaccinated videos.

Approach

The approach uses adversarial training. A 'vaccinator' adds imperceptible perturbations to the non-face region of an image, and a 'neutraliser' reconstructs the masked face region. The system is trained using a face masking attack, aiming for generality across different deepfake techniques.

Datasets

FaceForensics++

Model(s)

U-Net architecture with residual connections and multi-head attention mechanisms (for vaccinator and neutraliser); MLP, LeNet, ResNet, ViT, and ConvNeXT (for validator)

Author countries

Japan

← Previous