HQ-MPSD: A Multilingual Artifact-Controlled Benchmark for Partial Deepfake Speech Detection

Authors: Menglu Li, Majd Alber, Ramtin Asgarianamiri, Lian Zhao, Xiao-Ping Zhang

Published: 2025-12-15 06:18:43+00:00

Comment: 6 pages, 4 figures, 2 tables

AI Summary

This paper introduces HQ-MPSD, a high-quality, multilingual partial deepfake speech dataset designed to address limitations of existing datasets which often contain superficial artifacts. HQ-MPSD uses linguistically coherent splice points derived from forced alignment and incorporates background effects, yielding perceptually natural samples. Benchmarking state-of-the-art models on HQ-MPSD reveals significant performance drops, highlighting generalization challenges once low-level artifacts are removed and multilingual and acoustic diversity are introduced.

Abstract

Detecting partial deepfake speech is challenging because manipulations occur only in short regions while the surrounding audio remains authentic. However, existing detection methods are fundamentally limited by the quality of available datasets, many of which rely on outdated synthesis systems and generation procedures that introduce dataset-specific artifacts rather than realistic manipulation cues. To address this gap, we introduce HQ-MPSD, a high-quality multilingual partial deepfake speech dataset. HQ-MPSD is constructed using linguistically coherent splice points derived from fine-grained forced alignment, preserving prosodic and semantic continuity and minimizing audible and visual boundary artifacts. The dataset contains 350.8 hours of speech across eight languages and 550 speakers, with background effects added to better reflect real-world acoustic conditions. MOS evaluations and spectrogram analysis confirm the high perceptual naturalness of the samples. We benchmark state-of-the-art detection models through cross-language and cross-dataset evaluations, and all models experience performance drops exceeding 80% on HQ-MPSD. These results demonstrate that HQ-MPSD exposes significant generalization challenges once low-level artifacts are removed and multilingual and acoustic diversity are introduced, providing a more realistic and demanding benchmark for partial deepfake detection. The dataset can be found at: https://zenodo.org/records/17929533.

Key findings

State-of-the-art detection models experience performance drops exceeding 80% on HQ-MPSD in cross-language and cross-dataset evaluations. This severe degradation demonstrates that existing systems often overfit to superficial, dataset-specific artifacts (like unnatural boundary cues) and struggle to generalize to more realistic, artifact-controlled, multilingual, and acoustically diverse deepfake speech.

Approach

The authors create HQ-MPSD by generating fully deepfake speech from Multilingual LibriSpeech using XTTSv2, followed by a three-stage partial deepfake creation process. This process includes pre-normalization of loudness and spectral balance, alignment-based segment replacement using word-level forced alignment to ensure linguistic coherence, and acoustic augmentation with noise and reverberation to reflect real-world conditions. This meticulous design minimizes audible and visual boundary artifacts.

Datasets

HQ-MPSD (proposed dataset), Multilingual LibriSpeech (for generation), OpenSLR 26 (for RIR), MUSAN (for noise), PartialSpoof (for cross-dataset evaluation), Half-Truth, PartialEdit (for comparison in table).

Model(s)

GAT-ST (with SincNet, MFCC, spectrogram, W2v2-XLSR front-end features), TDAM, Nes2Net.

Author countries

China, Canada

← Previous