Improving Generalization Ability of Countermeasures for New Mismatch Scenario by Combining Multiple Advanced Regularization Terms

View on arXiv ← Back to list

Authors: Chang Zeng, Xin Wang, Xiaoxiao Miao, Erica Cooper, Junichi Yamagishi

Published: 2023-05-18 12:58:29+00:00

AI Summary

This paper addresses the generalization problem in audio deepfake detection when encountering unseen audio genres. It proposes a multi-task learning method that combines meta-optimization and genre alignment regularization to improve the generalization ability of countermeasure models. Experimental results demonstrate significant performance improvement compared to baseline systems in cross-genre scenarios.

Abstract

The ability of countermeasure models to generalize from seen speech synthesis methods to unseen ones has been investigated in the ASVspoof challenge. However, a new mismatch scenario in which fake audio may be generated from real audio with unseen genres has not been studied thoroughly. To this end, we first use five different vocoders to create a new dataset called CN-Spoof based on the CN-Celeb1&2 datasets. Then, we design two auxiliary objectives for regularization via meta-optimization and a genre alignment module, respectively, and combine them with the main anti-spoofing objective using learnable weights for multiple loss terms. The results on our cross-genre evaluation dataset for anti-spoofing show that the proposed method significantly improved the generalization ability of the countermeasures compared with the baseline system in the genre mismatch scenario.

Key findings

The proposed multi-task learning method significantly improved the generalization ability of the countermeasure model in cross-genre scenarios. The meta-optimization objective proved more effective than genre alignment in improving generalization. Combining both objectives further enhanced performance across various genres.

Approach

The authors create a new dataset, CN-Spoof, using various vocoders applied to CN-Celeb1&2. They propose a multi-task learning approach with three objectives: a main anti-spoofing objective, a meta-optimization objective simulating genre mismatch, and a genre alignment objective removing genre information. These objectives are combined with learnable weights.

Datasets

CN-Spoof (created by the authors using CN-Celeb1&2 and five vocoders: Multi-band MelGAN, Parallel WaveGAN, HiFiGAN, WORLD, and Griffin-Lim), CN-Celeb1&2

Model(s)

Lightweight Convolutional Neural Network (LCNN)

Author countries

Japan

← Previous