FractalForensics: Proactive Deepfake Detection and Localization via Fractal Watermarks

View on arXiv ← Back to list

Authors: Tianyi Wang, Harry Cheng, Ming-Hui Liu, Mohan Kankanhalli

Published: 2025-04-13 06:22:23+00:00

AI Summary

This paper introduces FractalForensics, a novel semi-fragile watermarking method for proactive deepfake detection and localization. It uses fractal watermarks generated by a parameter-driven pipeline, enabling both detection and localization of manipulations without requiring training on deepfakes.

Abstract

Proactive Deepfake detection via robust watermarks has been raised ever since passive Deepfake detectors encountered challenges in identifying high-quality synthetic images. However, while demonstrating reasonable detection performance, they lack localization functionality and explainability in detection results. Additionally, the unstable robustness of watermarks can significantly affect the detection performance accordingly. In this study, we propose novel fractal watermarks for proactive Deepfake detection and localization, namely FractalForensics. Benefiting from the characteristics of fractals, we devise a parameter-driven watermark generation pipeline that derives fractal-based watermarks and conducts one-way encryption regarding the parameters selected. Subsequently, we propose a semi-fragile watermarking framework for watermark embedding and recovery, trained to be robust against benign image processing operations and fragile when facing Deepfake manipulations in a black-box setting. Meanwhile, we introduce an entry-to-patch strategy that implicitly embeds the watermark matrix entries into image patches at corresponding positions, achieving localization of Deepfake manipulations. Extensive experiments demonstrate satisfactory robustness and fragility of our approach against common image processing operations and Deepfake manipulations, outperforming state-of-the-art semi-fragile watermarking algorithms and passive detectors for Deepfake detection. Furthermore, by highlighting the areas manipulated, our method provides explainability for the proactive Deepfake detection results.

Key findings

FractalForensics outperforms state-of-the-art semi-fragile watermarking algorithms and passive deepfake detectors in both detection and localization. It demonstrates robustness against common image processing operations and fragility against deepfake manipulations, generalizing well to unseen datasets and manipulation techniques. The method also shows sensitivity to image cropping.

Approach

FractalForensics embeds fractal watermarks into images using a deep learning framework. An entry-to-patch strategy maps watermark entries to image patches, enabling localization of manipulations. The method is trained without access to deepfake examples, working in a black-box setting.

Datasets

CelebA-HQ, LFW

Model(s)

A custom deep learning framework consisting of convolutional blocks, SEResBlocks (Squeeze-and-Excitation ResNet blocks), and a decoder for watermark embedding and recovery. A discriminator is also used for adversarial training.

Author countries

Singapore, China

← Previous