Can Current Detectors Catch Face-to-Voice Deepfake Attacks?

View on arXiv ← Back to list

Authors: Nguyen Linh Bao Nguyen, Alsharif Abuadbba, Kristen Moore, Tingming Wu

Published: 2025-10-23 21:24:55+00:00

AI Summary

This paper systematically evaluates the performance of state-of-the-art audio deepfake detectors against the novel Face-to-Voice (FOICE) attack, which generates synthetic speech from a single facial image. The study demonstrates that baseline detectors consistently fail to identify FOICE-generated audio, highlighting a security blind spot for cross-modal attacks. While targeted fine-tuning significantly improves FOICE detection, it often compromises the model's robustness and generalization ability against unseen generators like SpeechT5.

Abstract

The rapid advancement of generative models has enabled the creation of increasingly stealthy synthetic voices, commonly referred to as audio deepfakes. A recent technique, FOICE [USENIX'24], demonstrates a particularly alarming capability: generating a victim's voice from a single facial image, without requiring any voice sample. By exploiting correlations between facial and vocal features, FOICE produces synthetic voices realistic enough to bypass industry-standard authentication systems, including WeChat Voiceprint and Microsoft Azure. This raises serious security concerns, as facial images are far easier for adversaries to obtain than voice samples, dramatically lowering the barrier to large-scale attacks. In this work, we investigate two core research questions: (RQ1) can state-of-the-art audio deepfake detectors reliably detect FOICE-generated speech under clean and noisy conditions, and (RQ2) whether fine-tuning these detectors on FOICE data improves detection without overfitting, thereby preserving robustness to unseen voice generators such as SpeechT5. Our study makes three contributions. First, we present the first systematic evaluation of FOICE detection, showing that leading detectors consistently fail under both standard and noisy conditions. Second, we introduce targeted fine-tuning strategies that capture FOICE-specific artifacts, yielding significant accuracy improvements. Third, we assess generalization after fine-tuning, revealing trade-offs between specialization to FOICE and robustness to unseen synthesis pipelines. These findings expose fundamental weaknesses in today's defenses and motivate new architectures and training protocols for next-generation audio deepfake detection.

Key findings

Leading audio deepfake detectors exhibited clear blind spots against FOICE deepfakes, resulting in high Equal Error Rates (EERs) and inconsistent accuracy across conditions. Targeted fine-tuning yielded near-perfect performance on in-distribution FOICE samples, but this specialization often severely degraded performance on unseen synthesis pipelines like SpeechT5, confirming an inherent generalization trade-off. Only the detector designed for domain invariance (Ren et al.) maintained relatively stable cross-generator performance after fine-tuning.

Approach

The research benchmarked four distinct audio deepfake detection architectures using their baseline checkpoints against FOICE-generated speech under clean and noisy conditions. They then introduced targeted fine-tuning protocols using FOICE data to improve performance, followed by an assessment of the trade-off between specialization (to FOICE) and generalization (to an out-of-distribution pipeline, SpeechT5).

Datasets

VoxCeleb2, AVSpeech, FOICE (self-generated), SpeechT5 DS

Model(s)

AASIST, Ren et al. [15], Sun et al. [18], Temporal–Channel Modeling (TCM) [19]

Author countries

Australia

← Previous