Qualitative Failures of Image Generation Models and Their Application in Detecting Deepfakes

Authors: Ali Borji

Published: 2023-03-29 15:26:44+00:00

AI Summary

This research paper identifies qualitative shortcomings in image generation models, classifying them into five categories (human/animal body parts, geometry, physics, semantics/logic, text/noise/details). Understanding these failures helps improve model development and creates strategies for detecting deepfakes.

Abstract

The ability of image and video generation models to create photorealistic images has reached unprecedented heights, making it difficult to distinguish between real and fake images in many cases. However, despite this progress, a gap remains between the quality of generated images and those found in the real world. To address this, we have reviewed a vast body of literature from both academic publications and social media to identify qualitative shortcomings in image generation models, which we have classified into five categories. By understanding these failures, we can identify areas where these models need improvement, as well as develop strategies for detecting deep fakes. The prevalence of deep fakes in today's society is a serious concern, and our findings can help mitigate their negative impact.