Detecting Localized Deepfakes: How Well Do Synthetic Image Detectors Handle Inpainting?

Authors: Serafino Pandolfini, Lorenzo Pellegrini, Matteo Ferrara, Davide Maltoni

Published: 2025-12-18 15:54:51+00:00

Comment: 17 pages, 5 figures, 9 tables

AI Summary

This study systematically evaluates state-of-the-art deepfake detectors, originally trained on fully synthetic images, for their ability to detect localized image inpainting. The findings indicate that these models exhibit partial transferability to inpainting-based edits, reliably detecting medium to large-area manipulations and regeneration-style inpainting, often outperforming existing ad hoc methods.

Abstract

The rapid progress of generative AI has enabled highly realistic image manipulations, including inpainting and region-level editing. These approaches preserve most of the original visual context and are increasingly exploited in cybersecurity-relevant threat scenarios. While numerous detectors have been proposed for identifying fully synthetic images, their ability to generalize to localized manipulations remains insufficiently characterized. This work presents a systematic evaluation of state-of-the-art detectors, originally trained for the deepfake detection on fully synthetic images, when applied to a distinct challenge: localized inpainting detection. The study leverages multiple datasets spanning diverse generators, mask sizes, and inpainting techniques. Our experiments show that models trained on a large set of generators exhibit partial transferability to inpainting-based edits and can reliably detect medium- and large-area manipulations or regeneration-style inpainting, outperforming many existing ad hoc detection approaches.

Key findings

Models trained on a wide range of generators show competitive transferability to localized inpainting detection, performing well on medium to large manipulated areas (>50%) or full image regeneration. However, detection performance significantly degrades for small manipulated regions (<20%) or when facing high-quality generative models like Flux and Firefly families, indicating challenges for current classification-based detectors without explicit localization mechanisms.

Approach

The authors systematically evaluate state-of-the-art deepfake detectors, specifically ResNet-50 CLIP, DINOv2, and DINOv3, which were trained for full synthetic image detection on AI-GenBench. They assess the robustness and generalization of these models to localized image inpainting by testing them across diverse generators, mask sizes, and inpainting techniques without providing spatial supervision.

Datasets

BR-Gen, TGIF, TGIF-2, COCO2017, ImageNet, Places

Model(s)

UNKNOWN

Author countries

Italy

← Previous