MOSAIC: Interpretable Multi-Token Cross-Attention of Biophonetic and Self-Supervised Representations for Unified Voice Anti-Spoofing

Authors: Yugwon Won

Published: 2026-07-05 14:02:09+00:00

Comment: 5 pages, 2 figures. Submitted to IEEE Signal Processing Letters

AI Summary

MOSAIC is an interpretable multi-token cross-attention framework for unified voice anti-spoofing that combines biophonetic features and self-supervised representations. It leverages a 6x13 attention matrix to visualize cue-to-layer alignment and exhibits distinct activation patterns for bona fide and spoofed speech. This approach achieves competitive performance on ASVspoof 2019 LA/PA and ASVspoof 2021 LA/DF/PA benchmarks.

Abstract

The dominant trend in voice anti-spoofing fuses self-supervised (SSL) backbones (e.g., WavLM) with handcrafted features, yet such fusion typically lacks transparency in cue-to-layer interactions, and simple concatenation limits cross-modal learning. We propose MOSAIC (Multi-token Oriented Speech Anti-spoofing via Integrated Cross-attention), an interpretable multi-token cross-attention framework that splits a 152-dimensional biophonetic feature vector into six semantic-group query tokens (Praat, phase, LFCC mean/std, sub-band mean/std) and attends them over thirteen mean-std pooled WavLM-Large transformer layers as keys/values. The resulting 6x13 attention matrix visualizes cue-to-layer alignment; a z-score analysis of the per-token activations shows that biophonetic/phase tokens activate more on bona fide speech while spectral/channel tokens activate more on spoofed speech -- yielding per-cue, per-layer attribution that extends prior fusion approaches. Trained jointly with focal loss, a dual LA/PA domain-adversarial classifier, and a bona-fide-only VAE regularizer, MOSAIC attains EER 1.93% / 1.98% on ASVspoof 2019 LA / PA -- a single unified model that approaches the PA-specialized SOTA (LFCC-CMR, 1.34%) while remaining competitive on LA -- and 9.28% / 6.21% / 40.09% on ASVspoof 2021 LA / DF / PA.