What to Remember: Self-Adaptive Continual Learning for Audio Deepfake Detection

View on arXiv ← Back to list

Authors: Xiaohui Zhang, Jiangyan Yi, Chenglong Wang, Chuyuan Zhang, Siding Zeng, Jianhua Tao

Published: 2023-12-15 09:52:17+00:00

AI Summary

This paper proposes Radian Weight Modification (RWM), a continual learning approach for audio deepfake detection. RWM categorizes audio classes based on feature distribution compactness to adapt gradient modification directions, improving knowledge acquisition and mitigating forgetting when encountering new deepfake types.

Abstract

The rapid evolution of speech synthesis and voice conversion has raised substantial concerns due to the potential misuse of such technology, prompting a pressing need for effective audio deepfake detection mechanisms. Existing detection models have shown remarkable success in discriminating known deepfake audio, but struggle when encountering new attack types. To address this challenge, one of the emergent effective approaches is continual learning. In this paper, we propose a continual learning approach called Radian Weight Modification (RWM) for audio deepfake detection. The fundamental concept underlying RWM involves categorizing all classes into two groups: those with compact feature distributions across tasks, such as genuine audio, and those with more spread-out distributions, like various types of fake audio. These distinctions are quantified by means of the in-class cosine distance, which subsequently serves as the basis for RWM to introduce a trainable gradient modification direction for distinct data types. Experimental evaluations against mainstream continual learning methods reveal the superiority of RWM in terms of knowledge acquisition and mitigating forgetting in audio deepfake detection. Furthermore, RWM's applicability extends beyond audio deepfake detection, demonstrating its potential significance in diverse machine learning domains such as image recognition.

Key findings

RWM outperformed mainstream continual learning methods (EWC, LwF, OWM, DFWF) in audio deepfake detection across multiple datasets, showing superior knowledge acquisition and forgetting mitigation. It also demonstrated effectiveness in image recognition on the CLEAR benchmark, though performance was initially lower than some methods before experience 8.

Approach

RWM categorizes classes into groups with compact and spread-out feature distributions. It uses in-class cosine distance to guide a trainable gradient modification direction, minimizing interference with previously learned knowledge for similar classes and ensuring orthogonality for dissimilar classes.

Datasets

ASVspoof2019LA, ASVspoof2015, In-the-Wild, CLEAR (image recognition benchmark)

Model(s)

Wav2vec 2.0 (feature extractor), self-attention convolutional neural network (S-CNN) (classifier)

Author countries

China

← Previous