Social Media Authentication and Combating Deepfakes using Semi-fragile Invisible Image Watermarking

View on arXiv ← Back to list

Authors: Aakash Varma Nadimpalli, Ajita Rattani

Published: 2024-10-02 18:05:03+00:00

AI Summary

This paper proposes a novel semi-fragile invisible image watermarking technique for deepfake detection. The method embeds a 64-bit secret message into real images, making it robust to benign image processing but fragile to facial manipulations, achieving state-of-the-art performance.

Abstract

With the significant advances in deep generative models for image and video synthesis, Deepfakes and manipulated media have raised severe societal concerns. Conventional machine learning classifiers for deepfake detection often fail to cope with evolving deepfake generation technology and are susceptible to adversarial attacks. Alternatively, invisible image watermarking is being researched as a proactive defense technique that allows media authentication by verifying an invisible secret message embedded in the image pixels. A handful of invisible image watermarking techniques introduced for media authentication have proven vulnerable to basic image processing operations and watermark removal attacks. In response, we have proposed a semi-fragile image watermarking technique that embeds an invisible secret message into real images for media authentication. Our proposed watermarking framework is designed to be fragile to facial manipulations or tampering while being robust to benign image-processing operations and watermark removal attacks. This is facilitated through a unique architecture of our proposed technique consisting of critic and adversarial networks that enforce high image quality and resiliency to watermark removal efforts, respectively, along with the backbone encoder-decoder and the discriminator networks. Thorough experimental investigations on SOTA facial Deepfake datasets demonstrate that our proposed model can embed a $64$-bit secret as an imperceptible image watermark that can be recovered with a high-bit recovery accuracy when benign image processing operations are applied while being non-recoverable when unseen Deepfake manipulations are applied. In addition, our proposed watermarking technique demonstrates high resilience to several white-box and black-box watermark removal attacks. Thus, obtaining state-of-the-art performance.

Key findings

The proposed model achieves high imperceptibility (high PSNR and SSIM) and high bit recovery accuracy (BRA) under benign transformations. It demonstrates low BRA under unseen malicious transformations (Deepfakes) and is robust to white-box and black-box watermark removal attacks.

Approach

The authors propose a semi-fragile watermarking scheme using a U-Net encoder-decoder architecture, incorporating critic and adversarial networks to enhance image quality and watermark resilience against removal attacks. The watermark is designed to be fragile to facial manipulations while robust to benign image processing.

Datasets

FaceForensics++, CelebA, IMDB-WIKI

Model(s)

U-Net based encoder-decoder with critic and adversarial networks

Author countries

USA, USA

← Previous