Deepfakes for Medical Video De-Identification: Privacy Protection and Diagnostic Information Preservation

Authors: Bingquan Zhu, Hao Fang, Yanan Sui, Luming Li

Published: 2020-02-07 22:36:48+00:00

Comment: Accepted for publication at the AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES) 2020

AI Summary

This paper proposes using deepfake face-swapping technology for de-identifying patients in medical videos, aiming to protect privacy while preserving critical diagnostic information, such as facial and body keypoints. The approach demonstrates that face-swapping reliably de-identifies subjects and maintains keypoint invariability significantly better than traditional de-identification methods like masking or blurring. This facilitates ethical medical video data sharing for research.

Abstract

Data sharing for medical research has been difficult as open-sourcing clinical data may violate patient privacy. Traditional methods for face de-identification wipe out facial information entirely, making it impossible to analyze facial behavior. Recent advancements on whole-body keypoints detection also rely on facial input to estimate body keypoints. Both facial and body keypoints are critical in some medical diagnoses, and keypoints invariability after de-identification is of great importance. Here, we propose a solution using deepfake technology, the face swapping technique. While this swapping method has been criticized for invading privacy and portraiture right, it could conversely protect privacy in medical video: patients' faces could be swapped to a proper target face and become unrecognizable. However, it remained an open question that to what extent the swapping de-identification method could affect the automatic detection of body keypoints. In this study, we apply deepfake technology to Parkinson's disease examination videos to de-identify subjects, and quantitatively show that: face-swapping as a de-identification approach is reliable, and it keeps the keypoints almost invariant, significantly better than traditional methods. This study proposes a pipeline for video de-identification and keypoint preservation, clearing up some ethical restrictions for medical data sharing. This work could make open-source high quality medical video datasets more feasible and promote future medical research that benefits our society.

Key findings

The face-swapping method reliably de-identifies subjects, as quantified by dlib's face recognition, with swapped faces being distinct from original ones and similar to the target face. It significantly preserves keypoint information, achieving much higher Object Keypoint Similarity (OKS) scores (AP0.5:0.95 of 0.993) compared to masking (0.933) and blurring (0.916), particularly for head keypoints. The method is also deemed irreversible, protecting the original identity.

Approach

The proposed pipeline involves selecting patient videos (subject X) and an open-source target character's videos (subject Y), then training a Faceswap model to swap subject X's face with subject Y's. Control groups using traditional masking and blurring are created for comparison. Finally, Openpose is used to detect keypoints on original, swapped, masked, and blurred videos, while dlib's face recognition tool quantifies de-identification reliability.

Datasets

Parkinson's disease examination videos (of two subjects, M and F) and open-source videos of actor 01 (subject A) from the Deep Fake Detection Dataset by Google and JigSaw.

Model(s)

Faceswap (using a shared encoder and two decoders for face swapping), Openpose (employing Convolutional Pose Machine for keypoint detection), and dlib's face recognition tool (a 29-layer ResNet network, a revised ResNet-34) for de-identification quantification.

Author countries

China

← Previous