Generalizable Audio Deepfake Detection via Latent Space Refinement and Augmentation

View on arXiv ← Back to list

Authors: Wen Huang, Yanmei Gu, Zhiming Wang, Huijia Zhu, Yanmin Qian

Published: 2025-01-24 04:54:08+00:00

AI Summary

This paper proposes a novel audio deepfake detection strategy integrating Latent Space Refinement (LSR) and Latent Space Augmentation (LSA) to improve generalization. LSR uses multiple learnable prototypes for spoofed audio, while LSA augments data in the latent space, enhancing the model's ability to handle unseen attacks.

Abstract

Advances in speech synthesis technologies, like text-to-speech (TTS) and voice conversion (VC), have made detecting deepfake speech increasingly challenging. Spoofing countermeasures often struggle to generalize effectively, particularly when faced with unseen attacks. To address this, we propose a novel strategy that integrates Latent Space Refinement (LSR) and Latent Space Augmentation (LSA) to improve the generalization of deepfake detection systems. LSR introduces multiple learnable prototypes for the spoof class, refining the latent space to better capture the intricate variations within spoofed data. LSA further diversifies spoofed data representations by applying augmentation techniques directly in the latent space, enabling the model to learn a broader range of spoofing patterns. We evaluated our approach on four representative datasets, i.e. ASVspoof 2019 LA, ASVspoof 2021 LA and DF, and In-The-Wild. The results show that LSR and LSA perform well individually, and their integration achieves competitive results, matching or surpassing current state-of-the-art methods.

Key findings

LSR and LSA individually improve performance, and their combination achieves state-of-the-art results across multiple datasets. The approach outperforms or matches existing methods, demonstrating its effectiveness in generalizing across diverse deepfake audio detection tasks. Latent space augmentation generally outperforms input space augmentation.

Approach

The authors improve deepfake detection generalization by using Latent Space Refinement (LSR) with multiple learnable prototypes for spoofed audio and Latent Space Augmentation (LSA) to diversify spoofed data representations. This approach refines the latent space and expands it to better capture variations in spoofed audio.

Datasets

ASVspoof 2019 LA, ASVspoof 2021 LA, ASVspoof 2021 DF, In-The-Wild

Model(s)

Wav2Vec2.0 XLSR (frontend), AASIST (backend)

Author countries

China

← Previous