Synthetic Art Generation and DeepFake Detection A Study on Jamini Roy Inspired Dataset

View on arXiv ← Back to list

Authors: Kushal Agrawal, Romi Banerjee

Published: 2025-03-29 21:12:16+00:00

AI Summary

This research creates a new dataset of Jamini Roy-inspired artwork, including real and AI-generated images using Stable Diffusion 3, ControlNet, and IPAdapter. It then analyzes these images using qualitative, quantitative, and frequency-domain methods to identify subtle differences and assess the performance of state-of-the-art deepfake detection models on this culturally specific dataset.

Abstract

The intersection of generative AI and art is a fascinating area that brings both exciting opportunities and significant challenges, especially when it comes to identifying synthetic artworks. This study takes a unique approach by examining diffusion-based generative models in the context of Indian art, specifically focusing on the distinctive style of Jamini Roy. To explore this, we fine-tuned Stable Diffusion 3 and used techniques like ControlNet and IPAdapter to generate realistic images. This allowed us to create a new dataset that includes both real and AI-generated artworks, which is essential for a detailed analysis of what these models can produce. We employed various qualitative and quantitative methods, such as Fourier domain assessments and autocorrelation metrics, to uncover subtle differences between synthetic images and authentic pieces. A key takeaway from recent research is that existing methods for detecting deepfakes face considerable challenges, especially when the deepfakes are of high quality and tailored to specific cultural contexts. This highlights a critical gap in current detection technologies, particularly in light of the challenges identified above, where high-quality and culturally specific deepfakes are difficult to detect. This work not only sheds light on the increasing complexity of generative models but also sets a crucial foundation for future research aimed at effective detection of synthetic art.

Key findings

ControlNet and IPAdapter improve the quality of generated images, making them harder to detect as deepfakes. Existing deepfake detection models show varying performance on this culturally specific dataset, highlighting the need for more robust and adaptable methods. Frequency-domain analysis reveals persistent artifacts even in high-quality synthetic images.

Approach

The researchers fine-tuned Stable Diffusion 3 with ControlNet and IPAdapter to generate Jamini Roy-style images. They then used qualitative, quantitative (MSE, SSIM, PSNR, histogram correlation), and frequency-domain analysis to identify differences between real and synthetic images and evaluated the performance of three state-of-the-art deepfake detection models on the resulting dataset.

Datasets

A newly created dataset of Jamini Roy-inspired images, including both real and AI-generated images at various noise levels (0.0, 0.25, 0.5, 0.75) with and without ControlNet and IPAdapter.

Model(s)

Stable Diffusion 3, ControlNet, IPAdapter. Three state-of-the-art deepfake detection models (Tan et al. 2024, Corvi et al. 2023, Ojha et al. 2024) were used for evaluation.

Author countries

India

← Previous