DistilDIRE: A Small, Fast, Cheap and Lightweight Diffusion Synthesized Deepfake Detection

View on arXiv ← Back to list

Authors: Yewon Lim, Changyeon Lee, Aerin Kim, Oren Etzioni

Published: 2024-06-02 20:22:38+00:00

AI Summary

DistilDIRE is a fast and lightweight deepfake detection model that leverages knowledge distillation from a pre-trained diffusion model (DIRE) to significantly reduce computational demands without sacrificing performance. It achieves a 3.2 times faster inference speed than DIRE while maintaining robust detection capabilities.

Abstract

A dramatic influx of diffusion-generated images has marked recent years, posing unique challenges to current detection technologies. While the task of identifying these images falls under binary classification, a seemingly straightforward category, the computational load is significant when employing the reconstruction then compare technique. This approach, known as DIRE (Diffusion Reconstruction Error), not only identifies diffusion-generated images but also detects those produced by GANs, highlighting the technique's broad applicability. To address the computational challenges and improve efficiency, we propose distilling the knowledge embedded in diffusion models to develop rapid deepfake detection models. Our approach, aimed at creating a small, fast, cheap, and lightweight diffusion synthesized deepfake detector, maintains robust performance while significantly reducing operational demands. Maintaining performance, our experimental results indicate an inference speed 3.2 times faster than the existing DIRE framework. This advance not only enhances the practicality of deploying these systems in real-world settings but also paves the way for future research endeavors that seek to leverage diffusion model knowledge.

Key findings

DistilDIRE achieves comparable accuracy to DIRE but with a 3.2x faster inference speed and significantly reduced computational cost (approximately 97% reduction in FLOPS). The model effectively detects both diffusion-generated and GAN-generated deepfakes.

Approach

The approach distills knowledge from a pre-trained DIRE model into a smaller student model. This student model is trained using a combination of classification loss and knowledge distillation loss, incorporating the first-time step noise from a pre-trained diffusion model as input to improve detection accuracy.

Datasets

DiffusionForensics dataset (subsets from ImageNet and CelebA-HQ)

Model(s)

ResNet-50 (pre-trained on ImageNet-1K) as a teacher model; a student model trained from scratch for binary classification.

Author countries

Republic of Korea, United States

← Previous