Anomaly Detection and Localization for Speech Deepfakes via Feature Pyramid Matching

View on arXiv ← Back to list

Authors: Emma Coletta, Davide Salvi, Viola Negroni, Daniele Ugo Leonzio, Paolo Bestagini

Published: 2025-03-23 11:15:22+00:00

AI Summary

This paper introduces an interpretable one-class detection framework for speech deepfake detection, addressing limitations of supervised learning methods. The model, trained solely on real speech, identifies synthetic audio as anomalies and generates anomaly maps highlighting anomalous regions in time and frequency domains.

Abstract

The rise of AI-driven generative models has enabled the creation of highly realistic speech deepfakes - synthetic audio signals that can imitate target speakers' voices - raising critical security concerns. Existing methods for detecting speech deepfakes primarily rely on supervised learning, which suffers from two critical limitations: limited generalization to unseen synthesis techniques and a lack of explainability. In this paper, we address these issues by introducing a novel interpretable one-class detection framework, which reframes speech deepfake detection as an anomaly detection task. Our model is trained exclusively on real speech to characterize its distribution, enabling the classification of out-of-distribution samples as synthetically generated. Additionally, our framework produces interpretable anomaly maps during inference, highlighting anomalous regions across both time and frequency domains. This is done through a Student-Teacher Feature Pyramid Matching system, enhanced with Discrepancy Scaling to improve generalization capabilities across unseen data distributions. Extensive evaluations demonstrate the superior performance of our approach compared to the considered baselines, validating the effectiveness of framing speech deepfake detection as an anomaly detection problem.

Key findings

The proposed one-class anomaly detection framework outperforms supervised learning baselines and existing one-class methods across multiple datasets. The method demonstrates superior generalization capabilities to unseen synthesis techniques. Anomaly maps provide interpretable localization of synthetic artifacts.

Approach

The approach reframes speech deepfake detection as an anomaly detection problem using a Student-Teacher Feature Pyramid Matching (STFPM) system. A teacher network, pre-trained on speaker identification, and a student network are used; discrepancies between their activations on unseen data indicate deepfakes. Discrepancy Scaling enhances generalization.

Datasets

LibriSpeech (for training), LJSpeech, TIMIT-TTS, Purdue speech dataset, MLAAD, ASVspoof 2019, In-the-Wild, FakeOrReal (for evaluation)

Model(s)

Modified ResNet18 architecture for both teacher and student networks in the STFPM system.

Author countries

Italy

← Previous