Detection of Real-time DeepFakes in Video Conferencing with Active Probing and Corneal Reflection

View on arXiv ← Back to list

Authors: Hui Guo, Xin Wang, Siwei Lyu

Published: 2022-10-21 23:31:17+00:00

AI Summary

This paper proposes an active forensic method for real-time deepfake detection in video conferencing. It uses a projected pattern on the screen, whose reflection in the participant's cornea is analyzed to distinguish real individuals from deepfakes. This approach requires no specialized hardware.

Abstract

The COVID pandemic has led to the wide adoption of online video calls in recent years. However, the increasing reliance on video calls provides opportunities for new impersonation attacks by fraudsters using the advanced real-time DeepFakes. Real-time DeepFakes pose new challenges to detection methods, which have to run in real-time as a video call is ongoing. In this paper, we describe a new active forensic method to detect real-time DeepFakes. Specifically, we authenticate video calls by displaying a distinct pattern on the screen and using the corneal reflection extracted from the images of the call participant's face. This pattern can be induced by a call participant displaying on a shared screen or directly integrated into the video-call client. In either case, no specialized imaging or lighting hardware is required. Through large-scale simulations, we evaluate the reliability of this approach under a range in a variety of real-world imaging scenarios.

Key findings

The proposed method effectively distinguishes real participants from deepfakes by analyzing corneal reflections of a projected pattern. The approach is robust to variations in ambient lighting and shows promise for real-time deepfake detection in video conferencing. Current implementation has a processing speed of one frame per 4 seconds.

Approach

The method projects a distinct pattern onto the screen during a video call. The reflection of this pattern in the participant's cornea is captured and analyzed. A lack of a matching pattern in the corneal reflection suggests a deepfake.

Datasets

A real-world dataset of video conferencing videos with real participants and their deepfake counterparts created using Avatarify and DeepFaceLive; a simulated dataset to evaluate the method under various conditions.

Model(s)

Avatarify and DeepFaceLive (for generating deepfakes); Dlib (for face detection and landmark extraction); no deep learning model is used for deepfake detection itself, instead relying on template matching (normalized cross-correlation).

Author countries

USA

← Previous