Securing Social Media Against Deepfakes using Identity, Behavioral, and Geometric Signatures

View on arXiv ← Back to list

Authors: Muhammad Umar Farooq, Awais Khan, Ijaz Ul Haq, Khalid Mahmood Malik

Published: 2024-12-07 01:17:21+00:00

AI Summary

This paper proposes a novel deepfake detection framework, DBaGNet, which integrates Deep Identity, Behavioral, and Geometric (DBaG) signatures for improved generalizability. DBaGNet utilizes a triplet loss objective to enhance representation learning and achieves significant performance gains over state-of-the-art methods across six benchmark datasets.

Abstract

Trust in social media is a growing concern due to its ability to influence significant societal changes. However, this space is increasingly compromised by various types of deepfake multimedia, which undermine the authenticity of shared content. Although substantial efforts have been made to address the challenge of deepfake content, existing detection techniques face a major limitation in generalization: they tend to perform well only on specific types of deepfakes they were trained on.This dependency on recognizing specific deepfake artifacts makes current methods vulnerable when applied to unseen or varied deepfakes, thereby compromising their performance in real-world applications such as social media platforms. To address the generalizability of deepfake detection, there is a need for a holistic approach that can capture a broader range of facial attributes and manipulations beyond isolated artifacts. To address this, we propose a novel deepfake detection framework featuring an effective feature descriptor that integrates Deep identity, Behavioral, and Geometric (DBaG) signatures, along with a classifier named DBaGNet. Specifically, the DBaGNet classifier utilizes the extracted DBaG signatures, leveraging a triplet loss objective to enhance generalized representation learning for improved classification. Specifically, the DBaGNet classifier utilizes the extracted DBaG signatures and applies a triplet loss objective to enhance generalized representation learning for improved classification. To test the effectiveness and generalizability of our proposed approach, we conduct extensive experiments using six benchmark deepfake datasets: WLDR, CelebDF, DFDC, FaceForensics++, DFD, and NVFAIR. Specifically, to ensure the effectiveness of our approach, we perform cross-dataset evaluations, and the results demonstrate significant performance gains over several state-of-the-art methods.

Key findings

The DBaGNet framework significantly outperforms state-of-the-art methods in deepfake detection across multiple datasets. Cross-dataset and cross-manipulation evaluations demonstrate the improved generalizability of the proposed approach. Ablation studies confirm the effectiveness of the combined DBaG features.

Approach

The approach uses a multidisciplinary feature descriptor (DBaG) combining deep identity features, behavioral features (blendshapes), and geometric features (golden ratio-based). A triplet loss-based classifier, DBaGNet, is trained on these features to learn discriminative embeddings for enhanced generalizability.

Datasets

WLDR, CelebDF, DFDC, FaceForensics++, DFD, and NVFAIR

Model(s)

DBaGNet (a triplet loss-based classifier using a customized MobileNetV2-like architecture and MLP-Mixer for feature extraction, and AdaFace for deep identity features)

Author countries

USA

← Previous