Deepfakes on Demand: the rise of accessible non-consensual deepfake image generators

View on arXiv ← Back to list

Authors: Will Hawkins, Chris Russell, Brent Mittelstadt

Published: 2025-05-06 15:00:59+00:00

AI Summary

This paper presents an empirical study analyzing the accessibility of deepfake image generators online. The researchers found a substantial increase in easily accessible deepfake models, primarily hosted on Civitai, with nearly 35,000 models downloaded almost 15 million times since November 2022.

Abstract

Advances in multimodal machine learning have made text-to-image (T2I) models increasingly accessible and popular. However, T2I models introduce risks such as the generation of non-consensual depictions of identifiable individuals, otherwise known as deepfakes. This paper presents an empirical study exploring the accessibility of deepfake model variants online. Through a metadata analysis of thousands of publicly downloadable model variants on two popular repositories, Hugging Face and Civitai, we demonstrate a huge rise in easily accessible deepfake models. Almost 35,000 examples of publicly downloadable deepfake model variants are identified, primarily hosted on Civitai. These deepfake models have been downloaded almost 15 million times since November 2022, with the models targeting a range of individuals from global celebrities to Instagram users with under 10,000 followers. Both Stable Diffusion and Flux models are used for the creation of deepfake models, with 96% of these targeting women and many signalling intent to generate non-consensual intimate imagery (NCII). Deepfake model variants are often created via the parameter-efficient fine-tuning technique known as low rank adaptation (LoRA), requiring as few as 20 images, 24GB VRAM, and 15 minutes of time, making this process widely accessible via consumer-grade computers. Despite these models violating the Terms of Service of hosting platforms, and regulation seeking to prevent dissemination, these results emphasise the pressing need for greater action to be taken against the creation of deepfakes and NCII.

Key findings

The vast majority (96%) of deepfake models targeted women. Many models showed intent to generate NCII, despite violating platform Terms of Service. The ease of creating deepfakes using techniques like LoRA, requiring minimal resources and time, contributes to the problem.

Approach

The study used metadata analysis of thousands of publicly downloadable text-to-image models from Hugging Face and Civitai. They analyzed model names, descriptions, and tags to identify deepfake models and assess their intended use, particularly for generating non-consensual intimate imagery (NCII).

Datasets

Hugging Face and Civitai model repositories; a subset of 15,349 models (Stable Diffusion and Flux variants) were further analyzed.

Model(s)

Stable Diffusion and Flux text-to-image models, often fine-tuned using LoRA.

Author countries

United Kingdom

← Previous