Complementing Handcrafted Features with Raw Waveform Using a Light-weight Auxiliary Model

View on arXiv ← Back to list

Authors: Zhongwei Teng, Quchen Fu, Jules White, Maria Powell, Douglas C. Schmidt

Published: 2021-09-06 23:32:10+00:00

AI Summary

This paper proposes an Auxiliary RawNet (ARNet) model to improve audio spoof detection accuracy by combining handcrafted features with features learned from raw waveforms. ARNet uses a lightweight auxiliary encoder to process raw waveforms, supplementing information in handcrafted features at a low computational cost.

Abstract

An emerging trend in audio processing is capturing low-level speech representations from raw waveforms. These representations have shown promising results on a variety of tasks, such as speech recognition and speech separation. Compared to handcrafted features, learning speech features via backpropagation provides the model greater flexibility in how it represents data for different tasks theoretically. However, results from empirical study shows that, in some tasks, such as voice spoof detection, handcrafted features are more competitive than learned features. Instead of evaluating handcrafted features and raw waveforms independently, this paper proposes an Auxiliary Rawnet model to complement handcrafted features with features learned from raw waveforms. A key benefit of the approach is that it can improve accuracy at a relatively low computational cost. The proposed Auxiliary Rawnet model is tested using the ASVspoof 2019 dataset and the results from this dataset indicate that a light-weight waveform encoder can potentially boost the performance of handcrafted-features-based encoders in exchange for a small amount of additional computational work.

Key findings

Adding the lightweight auxiliary encoder significantly improved performance, reducing both EER and min-tDCF by approximately 50% across different combinations of handcrafted features and main encoders. The best performance was achieved with CQT/ECAPA-TDNN, resulting in an EER of 1.11% and min-tDCF of 0.0364, while maintaining a reasonable increase in computational cost.

Approach

The proposed ARNet architecture incorporates a lightweight auxiliary encoder to process raw waveforms alongside a main encoder processing handcrafted features. The outputs of both encoders are concatenated and fed into a final encoder, effectively combining information from both sources. This approach aims to leverage the strengths of both handcrafted and learned features.

Datasets

ASVspoof 2019 logical access (LA) dataset

Model(s)

Auxiliary RawNet (ARNet), which combines a lightweight auxiliary encoder (using strided convolution, max-pooling, and GRU) processing raw waveforms with existing models such as Res2Net, XVector, and ECAPA-TDNN processing handcrafted features (Mel-spectrogram and CQT).

Author countries

China, USA

← Previous