Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation

View on arXiv ← Back to list

Authors: Hemlata Tak, Massimiliano Todisco, Xin Wang, Jee-weon Jung, Junichi Yamagishi, Nicholas Evans

Published: 2022-02-24 17:55:00+00:00

AI Summary

This paper investigates using a wav2vec 2.0 front-end with fine-tuning for speaker verification spoofing and deepfake detection. Despite pre-training only on bona fide data, the approach achieves the lowest equal error rates reported in the literature for ASVspoof 2021 Logical Access and Deepfake databases, further improved with data augmentation.

Abstract

The performance of spoofing countermeasure systems depends fundamentally upon the use of sufficiently representative training data. With this usually being limited, current solutions typically lack generalisation to attacks encountered in the wild. Strategies to improve reliability in the face of uncontrolled, unpredictable attacks are hence needed. We report in this paper our efforts to use self-supervised learning in the form of a wav2vec 2.0 front-end with fine tuning. Despite initial base representations being learned using only bona fide data and no spoofed data, we obtain the lowest equal error rates reported in the literature for both the ASVspoof 2021 Logical Access and Deepfake databases. When combined with data augmentation,these results correspond to an improvement of almost 90% relative to our baseline system.

Key findings

The wav2vec 2.0 front-end, combined with data augmentation and a self-attentive aggregation layer, significantly reduces the equal error rate (EER) for both ASVspoof 2021 Logical Access and Deepfake datasets. These results represent the lowest EERs reported to date for these datasets, demonstrating improved generalization to unseen attacks.

Approach

The authors leverage self-supervised learning with a wav2vec 2.0 model as a front-end, pre-trained on bona fide speech data only. This pre-trained model is then fine-tuned on the ASVspoof 2019 LA training set, combined with a back-end AASIST model and data augmentation techniques, to improve generalization to unseen spoofing attacks.

Datasets

ASVspoof 2019 LA (training and development), ASVspoof 2021 Logical Access, ASVspoof 2021 Deepfake

Model(s)

wav2vec 2.0 XLS-R (0.3B) model, AASIST (integrated spectro-temporal graph attention network), RawNet2 encoder

Author countries

France, Japan, South Korea

← Previous