Benchmarking Deepart Detection

View on arXiv ← Back to list

Authors: Yabin Wang, Zhiwu Huang, Xiaopeng Hong

Published: 2023-02-28 10:34:44+00:00

AI Summary

This paper introduces a deepart detection database (DDDB) containing conventional and deepfake artworks generated by five state-of-the-art models. It proposes benchmark evaluations and solutions for once-for-all and continual deepart detection, addressing challenges like the scarcity of negative samples in continual learning.

Abstract

Deepfake technologies have been blurring the boundaries between the real and unreal, likely resulting in malicious events. By leveraging newly emerged deepfake technologies, deepfake researchers have been making a great upending to create deepfake artworks (deeparts), which are further closing the gap between reality and fantasy. To address potentially appeared ethics questions, this paper establishes a deepart detection database (DDDB) that consists of a set of high-quality conventional art images (conarts) and five sets of deepart images generated by five state-of-the-art deepfake models. This database enables us to explore once-for-all deepart detection and continual deepart detection. For the two new problems, we suggest four benchmark evaluations and four families of solutions on the constructed DDDB. The comprehensive study demonstrates the effectiveness of the proposed solutions on the established benchmark dataset, which is capable of paving a way to more interesting directions of deepart detection. The constructed benchmark dataset and the source code will be made publicly available.

Key findings

Continual deepart detection methods generally outperform once-for-all methods. The proposed framework for rescuing rehearsal-free methods in challenging continual learning scenarios shows significant improvements. Deeparts are shown to be more photorealistic than earlier deepfakes, posing a greater detection challenge.

Approach

The paper creates a benchmark dataset (DDDB) of conventional and deepfake artworks. It then proposes and evaluates four benchmark scenarios and corresponding solutions for both once-for-all and continual deepart detection, tackling issues like the lack of negative samples in continual learning through methods like knowledge distillation and prompt tuning.

Datasets

DDDB (containing conventional art images from LAION-5B and deepfake art images generated by Stable Diffusion, DALL-E 2, Imagen, Midjourney, and Parti)

Model(s)

ResNet-50, Vision Transformer (ViT), and several continual learning methods (iCaRL, BiC, GEM, Coil, Foster, LwF, EWC, S-Prompts)

Author countries

China, Singapore, United Kingdom

← Previous