DFIL: Deepfake Incremental Learning by Exploiting Domain-invariant Forgery Clues

Authors: Kun Pan, Yin Yifang, Yao Wei, Feng Lin, Zhongjie Ba, Zhenguang Liu, ZhiBo Wang, Lorenzo Cavallaro, Kui Ren

Published: 2023-09-18 07:02:26+00:00

Comment: Accepted by ACMMM2023

AI Summary

This paper introduces DFIL, a novel incremental learning framework for deepfake detection designed to adapt models to new deepfake methods with limited samples while mitigating catastrophic forgetting. DFIL achieves this by learning domain-invariant representations through supervised contrastive learning and preserving past knowledge via multi-perspective knowledge distillation. A strategic replay set selection mechanism, incorporating both central and hard representative samples, further enhances domain-invariant learning and knowledge retention.

Abstract

The malicious use and widespread dissemination of deepfake pose a significant crisis of trust. Current deepfake detection models can generally recognize forgery images by training on a large dataset. However, the accuracy of detection models degrades significantly on images generated by new deepfake methods due to the difference in data distribution. To tackle this issue, we present a novel incremental learning framework that improves the generalization of deepfake detection models by continual learning from a small number of new samples. To cope with different data distributions, we propose to learn a domain-invariant representation based on supervised contrastive learning, preventing overfit to the insufficient new data. To mitigate catastrophic forgetting, we regularize our model in both feature-level and label-level based on a multi-perspective knowledge distillation approach. Finally, we propose to select both central and hard representative samples to update the replay set, which is beneficial for both domain-invariant representation learning and rehearsal-based knowledge preserving. We conduct extensive experiments on four benchmark datasets, obtaining the new state-of-the-art average forgetting rate of 7.01 and average accuracy of 85.49 on FF++, DFDC-P, DFD, and CDF2. Our code is released at https://github.com/DeepFakeIL/DFIL.

Key findings

The DFIL framework achieved state-of-the-art results with an average forgetting rate of 7.01 and an average accuracy of 85.49 across four benchmark deepfake datasets. It significantly outperforms existing incremental learning methods in mitigating catastrophic forgetting and generalizing to new deepfake methods, even when learning from a small number of new samples. The approach also substantially improved the generalization ability of deepfake detection models, obtaining higher AUC scores on subsequent tasks compared to conventional methods.

Approach

The DFIL framework addresses deepfake incremental learning by combining three main components: learning domain-invariant representations via supervised contrastive learning, preserving past knowledge through multi-perspective knowledge distillation (both soft label and feature-level), and employing a novel replay strategy. This strategy selects a balanced set of central and hard representative samples from previous tasks to update the replay set, which is then merged with new task samples for training.

Datasets

Faceforensics++ (FF++), DeepFake Detection Challenge (DFDC-P), Deepfake Detection (DFD), Celeb-DFv2 (CDF2)

Model(s)

Xception (as the main backbone), Resnet34, EfficientNet

Author countries

China, Singapore, United Kingdom

← Previous