Towards Robust Protective Perturbation against DeepFake Face Swapping

Authors: Hengyang Yao, Lin Li, Ke Sun, Jianing Qiu, Huiping Chen

Published: 2025-12-08 07:12:43+00:00

AI Summary

This paper addresses the fragility of protective perturbations against DeepFake face swapping, which often degrade under common image transformations. The authors conduct a systematic analysis of 30 transformations, revealing that standard uniform sampling over transformations (EOT) is suboptimal. They propose Expectation Over Learned distribution of Transformation (EOLT), a novel framework that uses a policy network with reinforcement learning to automatically prioritize critical transformations and generate adaptive, instance-specific perturbations, significantly improving robustness.

Abstract

DeepFake face swapping enables highly realistic identity forgeries, posing serious privacy and security risks. A common defence embeds invisible perturbations into images, but these are fragile and often destroyed by basic transformations such as compression or resizing. In this paper, we first conduct a systematic analysis of 30 transformations across six categories and show that protection robustness is highly sensitive to the choice of training transformations, making the standard Expectation over Transformation (EOT) with uniform sampling fundamentally suboptimal. Motivated by this, we propose Expectation Over Learned distribution of Transformation (EOLT), the framework to treat transformation distribution as a learnable component rather than a fixed design choice. Specifically, EOLT employs a policy network that learns to automatically prioritize critical transformations and adaptively generate instance-specific perturbations via reinforcement learning, enabling explicit modeling of defensive bottlenecks while maintaining broad transferability. Extensive experiments demonstrate that our method achieves substantial improvements over state-of-the-art approaches, with 26% higher average robustness and up to 30% gains on challenging transformation categories.

Key findings

EOLT achieved substantial improvements over state-of-the-art approaches, demonstrating 26% higher average robustness and up to 30% gains on challenging transformation categories like Stylization. The method also showed strong generalization to unseen transformations and data, with marginal performance gaps between seen and unseen data settings. The learned policy network effectively prioritizes beneficial transformations (e.g., blur, noise) while downweighting less effective ones, confirming the advantage of adaptive sampling.

Approach

The authors propose Expectation Over Learned distribution of Transformation (EOLT), which replaces the uniform prior over transformations in EOT with an optimized, learnable transformation distribution. A policy network, conditioned on the input image, learns to prioritize critical transformations via reinforcement learning, generating instance-specific perturbations that maximize disruption to face-swapping models under various transformations.

Datasets

FFHQ

Model(s)

Policy model backbones: ViT-B/16, WideResNet28-10 (WRN28-10), PreAct ResNet-18 (PRN18). Evaluated against face-swapping models: SimSwap, ReFace. Identity features extracted using ArcFace.

Author countries

UK, China, UAE

← Previous