Real-Time Deepfake Detection in the Real-World

View on arXiv ← Back to list

Authors: Bar Cavia, Eliahu Horwitz, Tal Reiss, Yedid Hoshen

Published: 2024-06-13 17:59:23+00:00

AI Summary

This paper introduces LaDeDa, a patch-based deepfake detection algorithm that achieves state-of-the-art performance by focusing on local image features. It also presents Tiny-LaDeDa, a highly efficient distilled version, and WildRF, a new dataset of real-world deepfakes from social media, highlighting the limitations of current simulated datasets.

Abstract

Recent improvements in generative AI made synthesizing fake images easy; as they can be used to cause harm, it is crucial to develop accurate techniques to identify them. This paper introduces Locally Aware Deepfake Detection Algorithm (LaDeDa), that accepts a single 9x9 image patch and outputs its deepfake score. The image deepfake score is the pooled score of its patches. With merely patch-level information, LaDeDa significantly improves over the state-of-the-art, achieving around 99% mAP on current benchmarks. Owing to the patch-level structure of LaDeDa, we hypothesize that the generation artifacts can be detected by a simple model. We therefore distill LaDeDa into Tiny-LaDeDa, a highly efficient model consisting of only 4 convolutional layers. Remarkably, Tiny-LaDeDa has 375x fewer FLOPs and is 10,000x more parameter-efficient than LaDeDa, allowing it to run efficiently on edge devices with a minor decrease in accuracy. These almost-perfect scores raise the question: is the task of deepfake detection close to being solved? Perhaps surprisingly, our investigation reveals that current training protocols prevent methods from generalizing to real-world deepfakes extracted from social media. To address this issue, we introduce WildRF, a new deepfake detection dataset curated from several popular social networks. Our method achieves the top performance of 93.7% mAP on WildRF, however the large gap from perfect accuracy shows that reliable real-world deepfake detection is still unsolved.

Key findings

LaDeDa achieves near-perfect results on simulated deepfake datasets but performs poorly on real-world data from social media. WildRF, a new dataset, better reflects real-world challenges. Tiny-LaDeDa offers a significant speedup with only a minor accuracy loss.

Approach

LaDeDa divides images into 9x9 patches, assigning each a deepfake score. These scores are pooled to produce an image-level score. Tiny-LaDeDa is a distilled, computationally efficient version of LaDeDa. WildRF, a new dataset, addresses the generalization gap of existing methods by using real-world deepfakes from social media.

Datasets

ForenSynth, UFD, WildRF (a new dataset created from Reddit, Twitter (X), and Facebook)

Model(s)

LaDeDa (ResNet50 variant), Tiny-LaDeDa (4 convolutional layers), CNNDet, PatchFor, CLIP, NPR

Author countries

Israel

← Previous