Evidence Subspace Projection: Measuring How Much Evidence Explains Deepfake Detection in Self-Supervised Speech Models

View on arXiv ← Back to list

Authors: Yixuan Xiao, Cheng-Wei Lin, Xin Wang, Yassine El Kheir, Arnab Das, Tim Polzehl, Sebastian Möller, Ngoc Thang Vu

Published: 2026-07-13 13:22:19+00:00

Comment: Accepted to Interspeech 2026

AI Summary

This paper introduces Evidence Subspace Projection (ESP), a novel method to quantitatively analyze how much different evidence factors (e.g., attack category, gender, silence structure) explain deepfake detection decisions in self-supervised learning (SSL) speech models. ESP maps both evidence factors and authenticity labels into a shared neuron activation space, allowing for the direct measurement of each factor's explanatory power. The method is used to evaluate SSL models under various training conditions, revealing insights into their internal workings and reliance on specific cues.

Abstract

Self-supervised learning (SSL) models are widely used as feature extractors for state-of-the-art audio deepfake detection, but it remains unclear how to directly and quantitatively connect what SSL models capture to detection decisions. To address this gap, we propose Evidence Subspace Projection, a method that represents both evidence factors (e.g., attack category, codec, gender, transmission) and authenticity labels in a shared space constructed from SSL models' neuron activation patterns. By projecting the decision vector onto each evidence subspace, we obtain a scalar ratio that quantifies the explanatory power of each evidence type. We evaluate SSL models in raw, fine-tuned, and post-trained settings on multiple datasets. The results confirm findings from established studies, validating the proposed method, and reveal new insights into model behavior.

Key findings

Frozen models rely on dataset-specific shortcuts like silence and show undesirable alignment between within-spoof variation and the detection axis. Fine-tuning reduces within-spoof alignment but can amplify signal-level shortcuts, especially with homogeneous training data. Post-training further suppresses most dependencies, though the silence shortcut remains persistent.

Approach

The approach involves constructing a shared space from the neuron activation patterns of SSL models. Both detection decisions and various evidence factors are represented as one-vs-rest contrast vectors in this space. By projecting the decision vector onto subspaces spanned by these evidence factors, the method quantifies the explanatory power of each factor as a scalar ratio.

Datasets

ASV19, ASV21LA, ASV21DF, ASV5, ITW

Model(s)

XLSR, HuBERT

Author countries

Germany, Japan

← Previous