Adaptive Frequency Learning in Two-branch Face Forgery Detection

View on arXiv ← Back to list

Authors: Neng Wang, Yang Bai, Kun Yu, Yong Jiang, Shu-tao Xia, Yan Wang

Published: 2022-03-27 14:25:52+00:00

AI Summary

This paper proposes AFD, a novel adaptive frequency learning framework for face forgery detection. AFD adaptively learns frequency decomposition using soft masks and incorporates frequency features into spatial clues via an attention module, improving performance over existing two-branch methods.

Abstract

Face forgery has attracted increasing attention in recent applications of computer vision. Existing detection techniques using the two-branch framework benefit a lot from a frequency perspective, yet are restricted by their fixed frequency decomposition and transform. In this paper, we propose to Adaptively learn Frequency information in the two-branch Detection framework, dubbed AFD. To be specific, we automatically learn decomposition in the frequency domain by introducing heterogeneity constraints, and propose an attention-based module to adaptively incorporate frequency features into spatial clues. Then we liberate our network from the fixed frequency transforms, and achieve better performance with our data- and task-dependent transform layers. Extensive experiments show that AFD generally outperforms.

Key findings

AFD consistently outperforms existing methods on both whole and domain-specific test sets from FaceForensics++. Ablation studies demonstrate the effectiveness of adaptive frequency decomposition, attention-based feature fusion, and adaptive frequency transforms.

Approach

The proposed AFD framework uses a two-branch architecture. It adaptively learns frequency decomposition using soft masks optimized with a triplet loss, fuses frequency and spatial features with an attention module, and replaces fixed frequency transforms with learnable layers for improved performance.

Datasets

FaceForensics++ dataset (specifically the C40 low-quality subset).

Model(s)

Pre-trained Xception network.

Author countries

China

← Previous