LoRAX: LoRA eXpandable Networks for Continual Synthetic Image Attribution

View on arXiv ← Back to list

Authors: Danielle Sullivan-Pao, Nicole Tian, Pooya Khorrami

Published: 2025-04-10 22:20:00+00:00

AI Summary

LoRAX is a parameter-efficient class incremental learning algorithm for synthetic image attribution that adapts to new generative image models without full retraining. It achieves this by training a low-rank adaptation (LoRA) feature extractor per task, requiring less than 3% of the parameters of a full-rank implementation.

Abstract

As generative AI image technologies become more widespread and advanced, there is a growing need for strong attribution models. These models are crucial for verifying the authenticity of images and identifying the architecture of their originating generative models-key to maintaining media integrity. However, attribution models struggle to generalize to unseen models, and traditional fine-tuning methods for updating these models have shown to be impractical in real-world settings. To address these challenges, we propose LoRA eXpandable Networks (LoRAX), a parameter-efficient class incremental algorithm that adapts to novel generative image models without the need for full retraining. Our approach trains an extremely parameter-efficient feature extractor per continual learning task via Low Rank Adaptation. Each task-specific feature extractor learns distinct features while only requiring a small fraction of the parameters present in the underlying feature extractor's backbone model. Our extensive experimentation shows LoRAX outperforms or remains competitive with state-of-the-art class incremental learning algorithms on the Continual Deepfake Detection benchmark across all training scenarios and memory settings, while requiring less than 3% of the number of trainable parameters per feature extractor compared to the full-rank implementation. LoRAX code is available at: https://github.com/mit-ll/lorax_cil.

Key findings

LoRAX outperforms or remains competitive with state-of-the-art class incremental learning algorithms on the CDDB benchmark across all training scenarios and memory settings. It achieves this while using significantly fewer parameters than other methods. The ConViT backbone consistently outperforms ResNet in continual learning tasks.

Approach

LoRAX uses Low Rank Adaptation (LoRA) to train a parameter-efficient feature extractor for each new generative model encountered. These task-specific extractors are concatenated to form a 'super feature' fed into a unified classifier. The underlying model is frozen, mitigating catastrophic forgetting.

Datasets

Continual Deepfake Detection Benchmark (CDDB)

Model(s)

ConViT (Base, Small, Tiny), ResNet (34, 50, 152), LoRA

Author countries

USA

← Previous