Attentive Merging of Hidden Embeddings from Pre-trained Speech Model for Anti-spoofing Detection

Authors: Zihan Pan, Tianchi Liu, Hardik B. Sailor, Qiongqiong Wang

Published: 2024-06-12 08:27:44+00:00

AI Summary

This paper investigates the multi-layer behavior of the WavLM self-supervised learning model for anti-spoofing detection and proposes an attentive merging method to leverage its hierarchical hidden embeddings. The approach demonstrates the feasibility of fine-tuning WavLM to achieve state-of-the-art Equal Error Rates (EERs) on ASVspoof datasets. A key finding is that early hidden transformer layers contribute significantly, allowing for computational efficiency by using only a partial pre-trained model.

Abstract

Self-supervised learning (SSL) speech representation models, trained on large speech corpora, have demonstrated effectiveness in extracting hierarchical speech embeddings through multiple transformer layers. However, the behavior of these embeddings in specific tasks remains uncertain. This paper investigates the multi-layer behavior of the WavLM model in anti-spoofing and proposes an attentive merging method to leverage the hierarchical hidden embeddings. Results demonstrate the feasibility of fine-tuning WavLM to achieve the best equal error rate (EER) of 0.65%, 3.50%, and 3.19% on the ASVspoof 2019LA, 2021LA, and 2021DF evaluation sets, respectively. Notably, We find that the early hidden transformer layers of the WavLM large model contribute significantly to anti-spoofing task, enabling computational efficiency by utilizing a partial pre-trained model.

Key findings

The fine-tuned WavLM with Attentive Merging (AttM-LSTM) achieved competitive EERs of 0.65%, 3.50%, and 3.19% on ASVspoof 2019LA, 2021LA, and 2021DF evaluation sets respectively, outperforming several state-of-the-art systems, especially on the challenging 2021DF dataset. It was found that early hidden transformer layers (up to about layer 12) of the WavLM large model contribute most significantly to anti-spoofing performance. This allows for superior performance with reduced computational resources by using only a subset of the pre-trained model's layers.

Approach

The authors propose an Attentive Merging (AttM) method that combines hidden embeddings from multiple transformer encoder layers of a pre-trained WavLM model. These attentively merged features are then fed into a downstream classifier, which can be either an LSTM or an ECAPA-TDNN, for anti-spoofing detection. The method also explores the effectiveness of using only a subset of the WavLM's transformer layers.

Datasets

ASVspoof 2019 LA (training, development, evaluation), ASVspoof 2021 LA (evaluation), ASVspoof 2021 DF (evaluation)

Model(s)

WavLM large (pre-trained speech representation model), Attentive Merging (AttM), Linear Merging (LinM), LSTM, ECAPA-TDNN

Author countries

Singapore

← Previous