QAMO: Quality-aware Multi-centroid One-class Learning For Speech Deepfake Detection

View on arXiv ← Back to list

Authors: Duc-Tuan Truong, Tianchi Liu, Ruijie Tao, Junjie Li, Kong Aik Lee, Eng Siong Chng

Published: 2025-09-25 02:27:49+00:00

AI Summary

QAMO (Quality-Aware Multi-Centroid One-Class Learning) is proposed to improve speech deepfake detection by addressing the oversimplification inherent in single-centroid one-class models. It introduces multiple quality-aware centroids, each optimized to represent a distinct speech quality subspace derived from estimated Mean Opinion Scores (MOS). This framework better models the intra-class variability of bona fide speech and utilizes a multi-centroid ensemble scoring strategy during inference to reduce the need for explicit quality labels.

Abstract

Recent work shows that one-class learning can detect unseen deepfake attacks by modeling a compact distribution of bona fide speech around a single centroid. However, the single-centroid assumption can oversimplify the bona fide speech representation and overlook useful cues, such as speech quality, which reflects the naturalness of the speech. Speech quality can be easily obtained using existing speech quality assessment models that estimate it through Mean Opinion Score. In this paper, we propose QAMO: Quality-Aware Multi-Centroid One-Class Learning for speech deepfake detection. QAMO extends conventional one-class learning by introducing multiple quality-aware centroids. In QAMO, each centroid is optimized to represent a distinct speech quality subspaces, enabling better modeling of intra-class variability in bona fide speech. In addition, QAMO supports a multi-centroid ensemble scoring strategy, which improves decision thresholding and reduces the need for quality labels during inference. With two centroids to represent high- and low-quality speech, our proposed QAMO achieves an equal error rate of 5.09% in In-the-Wild dataset, outperforming previous one-class and quality-aware systems.

Key findings

QAMO achieved superior performance compared to single-centroid baselines and previous quality-aware systems, achieving an EER of 5.09% on the challenging In-the-Wild dataset when integrated with XLSR-Conformer-TCM. Ablation studies confirmed that the quality classification loss is crucial for preventing centroid collapse and that the ensemble scoring strategy provided significantly more stable and discriminative score distributions than maximum-score inference.

Approach

QAMO extends conventional one-class learning by training multiple quality-aware centroids, where each centroid corresponds to a discrete speech quality level (e.g., high or low quality), utilizing MOS predictions to define these levels. The model is optimized using a combined loss function consisting of a QAMO one-class loss and an AM-Softmax quality classification loss to ensure centroid discriminability. At inference, QAMO computes the final countermeasure score by ensembling (averaging) the distances across all learned centroids.

Datasets

ASVspoof2019 Logical Access (LA), ASVspoof2021 LA, ASVspoof2021 DF, In-the-Wild (ITW), Fake-or-Real (FoR) norm-test subset.

Model(s)

XLSR-Conformer-TCM, XLSR-Nes2NetX (QAMO is implemented on these backbones).

Author countries

Singapore, Hong Kong

← Previous