Towards Data Drift Monitoring for Speech Deepfake Detection in the context of MLOps

Authors: Xin Wang, Wanying Ge, Junichi Yamagishi

Published: 2025-09-12 09:26:56+00:00

Comment: code to be pushed to https://github.com/nii-yamagishilab/AntiDeepfake

AI Summary

This paper investigates data drift monitoring for speech deepfake detection within an MLOps context. It explores whether drift from new text-to-speech (TTS) attacks can be monitored using feature distribution distances and if fine-tuning detectors with similarly drifted data can reduce this drift and improve detection performance. The study demonstrates that drift can be monitored and effectively reduced by fine-tuning, leading to improved detection error rates.

Abstract

When being delivered in applications or services on the cloud, static speech deepfake detectors that are not updated will become vulnerable to newly created speech deepfake attacks. From the perspective of machine learning operations (MLOps), this paper tries to answer whether we can monitor new and unseen speech deepfake data that drifts away from a seen reference data set. We further ask, if drift is detected, whether we can fine-tune the detector using similarly drifted data, reduce the drift, and improve the detection performance. On a toy dataset and the large-scale MLAAD dataset, we show that the drift caused by new text-to-speech (TTS) attacks can be monitored using distances between the distributions of the new data and reference data. Furthermore, we demonstrate that fine-tuning the detector using data generated by the new TTS deepfakes can reduce the drift and the detection error rates.

Key findings

The study found that data drift caused by new text-to-speech (TTS) attacks can be effectively monitored using distances between feature distributions, with newer TTS systems causing higher drift values. Fine-tuning the detectors with data from these new TTS attacks significantly reduces both the observed data drift and the detection error rates. The reduction in drift and error rates is more pronounced with larger amounts of fine-tuning data, and fine-tuning with data from similar attacks also yields benefits.

Approach

The authors monitor data drift by measuring the distance between feature distributions of incoming test data (from new TTS attacks) and a reference dataset, using metrics like Wasserstein-1, K-S, and KLD. Upon detecting drift, they fine-tune pre-trained deepfake detectors with data generated by the new TTS attacks. This process aims to reduce the observed data drift and concurrently improve the detector's performance.

Datasets

['LJSpeech-TTS dataset', 'MLAAD (version 7.0)', 'ASVspoof 2019 development set', 'M-AILABS']

Model(s)

['AASIST', 'Wav2vec 2.0 (W2V)', 'XLS-R (XSLR2b)']

Author countries

['Japan']

← Previous