DeepSonar: Towards Effective and Robust Detection of AI-Synthesized Fake Voices

Authors: Run Wang, Felix Juefei-Xu, Yihao Huang, Qing Guo, Xiaofei Xie, Lei Ma, Yang Liu

Published: 2020-05-28 04:02:52+00:00

Comment: Accepted by ACM MM'20

AI Summary

This paper introduces DeepSonar, a novel approach for detecting AI-synthesized fake voices by monitoring the layer-wise neuron behaviors of a speaker recognition (SR) deep neural network. The method leverages neuron activation patterns to capture subtle differences between real and fake voices, providing a cleaner signal for a binary classifier than raw audio inputs. Experiments across three datasets, including commercial products and different languages, demonstrate high detection accuracy (98.1% average) and robustness against manipulation attacks like voice conversion and additive real-world noises.

Abstract

With the recent advances in voice synthesis, AI-synthesized fake voices are indistinguishable to human ears and widely are applied to produce realistic and natural DeepFakes, exhibiting real threats to our society. However, effective and robust detectors for synthesized fake voices are still in their infancy and are not ready to fully tackle this emerging threat. In this paper, we devise a novel approach, named \\emph{DeepSonar}, based on monitoring neuron behaviors of speaker recognition (SR) system, \\ie, a deep neural network (DNN), to discern AI-synthesized fake voices. Layer-wise neuron behaviors provide an important insight to meticulously catch the differences among inputs, which are widely employed for building safety, robust, and interpretable DNNs. In this work, we leverage the power of layer-wise neuron activation patterns with a conjecture that they can capture the subtle differences between real and AI-synthesized fake voices, in providing a cleaner signal to classifiers than raw inputs. Experiments are conducted on three datasets (including commercial products from Google, Baidu, \\etc) containing both English and Chinese languages to corroborate the high detection rates (98.1\\% average accuracy) and low false alarm rates (about 2\\% error rate) of DeepSonar in discerning fake voices. Furthermore, extensive experimental results also demonstrate its robustness against manipulation attacks (\\eg, voice conversion and additive real-world noises). Our work further poses a new insight into adopting neuron behaviors for effective and robust AI aided multimedia fakes forensics as an inside-out approach instead of being motivated and swayed by various artifacts introduced in synthesizing fakes.

Key findings

DeepSonar achieved an impressive average accuracy of 98.1% and an equal error rate (EER) of less than 2% across three diverse datasets, covering both English and Chinese languages and various synthetic techniques. The approach also demonstrated strong robustness against manipulation attacks, including voice conversions (resampling, speed, pitch shifts) and additive real-world noises (indoor and outdoor environmental sounds), outperforming a state-of-the-art bispectral analysis baseline in both effectiveness and robustness.

Approach

DeepSonar operates by first feeding audio samples (real or fake) into a pre-trained DNN-based speaker recognition system. It then captures the layer-wise neuron behaviors, specifically focusing on 'activated neurons' determined by either average count neuron (ACN) or Top-k activated neuron (TKAN) criteria. These extracted neuron behavior patterns are vectorized and used as features to train a shallow neural network-based binary classifier to distinguish between real human voices and AI-synthesized fake voices.

Datasets

FoR (public dataset with voices synthesized by commercial products like Amazon AWS Polly, Google Cloud TTS, Microsoft Azure TTS), Sprocket-VC (self-built using 'sprocket' tool, real voices from VCC16 & VCC18), MC-TTS (self-built using Baidu speech synthesis system, Chinese language, real voices from 'lecture_tts'). ESC-50 was used for real-world noise samples in robustness evaluation.

Model(s)

A 'thin-ResNet' based deep neural network for the speaker recognition (SR) system. A shallow neural network with five fully-connected layers is used as the binary classifier for deepfake detection.

Author countries

Singapore, USA, China, Japan

← Previous