COMICS: End-to-end Bi-grained Contrastive Learning for Multi-face Forgery Detection

View on arXiv ← Back to list

Authors: Cong Zhang, Honggang Qi, Shuhui Wang, Yuezun Li, Siwei Lyu

Published: 2023-08-03 03:37:13+00:00

AI Summary

COMICS is an end-to-end multi-face forgery detection framework that uses a bi-grained contrastive learning approach. This approach captures forgery traces at both coarse-grained (proposal-wise) and fine-grained (pixel-wise) levels, improving detection accuracy compared to previous methods.

Abstract

DeepFakes have raised serious societal concerns, leading to a great surge in detection-based forensics methods in recent years. Face forgery recognition is a standard detection method that usually follows a two-phase pipeline. While those methods perform well in ideal experimental environment, they face challenges when dealing with DeepFakes in the wild involving complex background and multiple faces of varying sizes. Moreover, most face forgery recognition methods can only process one face at a time. One straightforward way to address this issue is to simultaneous process multi-face by integrating face extraction and forgery detection in an end-to-end fashion by adapting advanced object detection architectures. However, as these object detection architectures are designed to capture the discriminative features of different object categories rather than the subtle forgery traces among the faces, the direct adaptation suffers from limited representation ability. In this paper, we propose COMICS, an end-to-end framework for multi-face forgery detection. COMICS integrates face extraction and forgery detection in a seamless manner and adapts to advanced object detection architectures. The proposed bi-grained contrastive learning approach explores face forgery traces at both the coarse- and fine-grained levels. Specifically, coarse-grained level contrastive learning captures the discriminative features among positive and negative proposal pairs at multiple layers produced by the proposal generator, and fine-grained level contrastive learning captures the pixel-wise discrepancy between the forged and original areas of the same face and the pixel-wise content inconsistency among different faces. Extensive experiments on the OpenForensics and FFIW datasets demonstrate that our method outperforms other counterparts and shows great potential for being integrated into various architectures.

Key findings

COMICS outperforms other state-of-the-art methods on the OpenForensics dataset, especially in challenging scenarios. The bi-grained contrastive learning significantly improves detection accuracy, particularly for small objects. The method also shows improved performance on the FFIW dataset, although limitations exist due to the imperfect real face mask generation.

Approach

COMICS integrates face extraction and forgery detection in an end-to-end manner using object detection architectures. It employs a bi-grained contrastive learning strategy, leveraging both coarse-grained (proposal-level) and fine-grained (pixel-level) comparisons to learn discriminative features for detecting forgery traces.

Datasets

OpenForensics, FFIW

Model(s)

BlendMask (with ResNet50-FPN backbone), MaskRCNN, ConInst, MEInst, SOLOv2, Swin Transformer

Author countries

China, USA

← Previous