Deepfake histological images for enhancing digital pathology

View on arXiv ← Back to list

Authors: Kianoush Falahkheirkhah, Saumya Tiwari, Kevin Yeh, Sounak Gupta, Loren Herrera-Hernandez, Michael R. McCarthy, Rafael E. Jimenez, John C. Cheville, Rohit Bhargava

Published: 2022-06-16 17:11:08+00:00

AI Summary

This paper presents SHI-GAN, a generative adversarial network (GAN) that synthesizes realistic histological images from class labels. The model successfully generates high-resolution images of prostate and colon tissues, comparable to real data in training deep learning models for diagnosis and indistinguishable from real images to pathologists.

Abstract

An optical microscopic examination of thinly cut stained tissue on glass slides prepared from a FFPE tissue blocks is the gold standard for tissue diagnostics. In addition, the diagnostic abilities and expertise of any pathologist is dependent on their direct experience with common as well as rarer variant morphologies. Recently, deep learning approaches have been used to successfully show a high level of accuracy for such tasks. However, obtaining expert-level annotated images is an expensive and time-consuming task and artificially synthesized histological images can prove greatly beneficial. Here, we present an approach to not only generate histological images that reproduce the diagnostic morphologic features of common disease but also provide a user ability to generate new and rare morphologies. Our approach involves developing a generative adversarial network model that synthesizes pathology images constrained by class labels. We investigated the ability of this framework in synthesizing realistic prostate and colon tissue images and assessed the utility of these images in augmenting diagnostic ability of machine learning methods as well as their usability by a panel of experienced anatomic pathologists. Synthetic data generated by our framework performed similar to real data in training a deep learning model for diagnosis. Pathologists were not able to distinguish between real and synthetic images and showed a similar level of inter-observer agreement for prostate cancer grading. We extended the approach to significantly more complex images from colon biopsies and showed that the complex microenvironment in such tissues can also be reproduced. Finally, we present the ability for a user to generate deepfake histological images via a simple markup of sematic labels.

Key findings

Synthetic images generated by SHI-GAN performed similarly to real data in training deep learning models. Pathologists could not reliably distinguish between real and synthetic images and showed similar inter-observer agreement for prostate cancer grading. The model successfully generated high-resolution images of complex colon tissues.

Approach

The authors developed SHI-GAN, a conditional GAN that takes class labels as input and generates corresponding histological images. The model incorporates conditional normalization layers and utilizes perceptual and GAN losses for training. Multiple discriminators at different scales are used to enhance image realism.

Datasets

Publicly available prostate cancer dataset (102 slides, 20 biopsies used for training), and a second dataset of colon biopsies (8 tissue microarrays, 1mm diameter samples).

Model(s)

Generative Adversarial Network (GAN), specifically SHI-GAN, with a generator and multiple discriminators. A pre-trained VGG-19 was used for perceptual loss calculation. U-Net and ResNet based architectures were also used for semantic segmentation model evaluation.

Author countries

USA

← Previous