ManipShield: A Unified Framework for Image Manipulation Detection, Localization and Explanation

View on arXiv ← Back to list

Authors: Zitong Xu, Huiyu Duan, Xiaoyu Wang, Zhaolin Cai, Kaiwei Zhang, Qiang Hu, Jing Liu, Xiongkuo Min, Guangtao Zhai

Published: 2025-11-18 08:50:17+00:00

AI Summary

This paper introduces ManipShield, a unified framework for AI-edited image manipulation detection, localization, and explanation, built upon a new large-scale benchmark called ManipBench. ManipBench contains over 450K manipulated images generated by 25 state-of-the-art editing models, with 100K images further annotated with bounding boxes, judgment cues, and textual explanations for interpretable detection. ManipShield, an MLLM-based model, leverages contrastive LoRA fine-tuning and task-specific decoders to achieve state-of-the-art performance and strong generalization to unseen manipulation models.

Abstract

With the rapid advancement of generative models, powerful image editing methods now enable diverse and highly realistic image manipulations that far surpass traditional deepfake techniques, posing new challenges for manipulation detection. Existing image manipulation detection and localization (IMDL) benchmarks suffer from limited content diversity, narrow generative-model coverage, and insufficient interpretability, which hinders the generalization and explanation capabilities of current manipulation detection methods. To address these limitations, we introduce \\textbf{ManipBench}, a large-scale benchmark for image manipulation detection and localization focusing on AI-edited images. ManipBench contains over 450K manipulated images produced by 25 state-of-the-art image editing models across 12 manipulation categories, among which 100K images are further annotated with bounding boxes, judgment cues, and textual explanations to support interpretable detection. Building upon ManipBench, we propose \\textbf{ManipShield}, an all-in-one model based on a Multimodal Large Language Model (MLLM) that leverages contrastive LoRA fine-tuning and task-specific decoders to achieve unified image manipulation detection, localization, and explanation. Extensive experiments on ManipBench and several public datasets demonstrate that ManipShield achieves state-of-the-art performance and exhibits strong generality to unseen manipulation models. Both ManipBench and ManipShield will be released upon publication.

Key findings

ManipShield achieved state-of-the-art performance on the proposed ManipBench and several public image manipulation datasets. It demonstrated strong generalization capabilities to unseen manipulation models, outperforming existing methods. The unified framework also provides transparent detection by offering detailed textual explanations and localizing manipulated regions with high accuracy.

Approach

ManipShield is an all-in-one model based on a Multimodal Large Language Model (MLLM) like InternVL3.5. It employs contrastive LoRA fine-tuning on the vision encoder to discriminate manipulated images, uses a Layer Discrimination Selection (LDS) module to identify the most informative LLM layer, and utilizes task-specific decoders (for detection, unrealistic attribute extraction, and region localization) to interpret the hidden state from this layer, which are then integrated to generate explicit textual explanations.

Datasets

ManipBench (proposed), CASIA2, IMD2020, DDFT, DF40.

Model(s)

ManipShield (proposed), InternVL3.5 (backbone MLLM). Other models used for comparison include: ResNet50, Swin-T, MVSSNet, PSCCNet, HifiNet, FakeShield, CNNSpot, Lagrad, Univ, AIDE, LLaMA3.2-Vision, LLaVA-1.6, MiniCPM-V2.6, mPLUG-Owl3, CogVLM, Ovis2.5, DeepSeekVL2, InternVL3, Qwen2.5-VL, Qwen3-VL, Gemini2.5-Pro, ChatGPT-4o, Fast R-CNN, DETR, DeformableDETR, GroundingDINO.

Author countries

China

← Previous