Position: It's Time to Act on the Risk of Efficient Personalized Text Generation

View on arXiv ← Back to list

Authors: Eugenia Iofinova, Andrej Jovanovic, Dan Alistarh

Published: 2025-02-10 15:25:11+00:00

AI Summary

This position paper highlights the emerging risk of using personalized large language models (LLMs) for malicious impersonation. The accessibility of efficient fine-tuning techniques allows for the creation of realistic text deepfakes, posing significant threats such as phishing and character assassination.

Abstract

The recent surge in high-quality open-source Generative AI text models (colloquially: LLMs), as well as efficient finetuning techniques, have opened the possibility of creating high-quality personalized models that generate text attuned to a specific individual's needs and are capable of credibly imitating their writing style by refining an open-source model with that person's own data. The technology to create such models is accessible to private individuals, and training and running such models can be done cheaply on consumer-grade hardware. While these advancements are a huge gain for usability and privacy, this position paper argues that the practical feasibility of impersonating specific individuals also introduces novel safety risks. For instance, this technology enables the creation of phishing emails or fraudulent social media accounts, based on small amounts of publicly available text, or by the individuals themselves to escape AI text detection. We further argue that these risks are complementary to - and distinct from - the much-discussed risks of other impersonation attacks such as image, voice, or video deepfakes, and are not adequately addressed by the larger research community, or the current generation of open- and closed-source models.

Key findings

Personalized LLMs can convincingly imitate individuals' writing styles, enabling sophisticated impersonation attacks. Current AI-generated text detection tools are ineffective against personalized LLM outputs. Existing legal frameworks and model safety measures are insufficient to address these risks.

Approach

The authors argue that the ease and affordability of fine-tuning open-source LLMs with an individual's writing style creates a new and significant risk of text-based impersonation. They demonstrate the feasibility of creating personalized models capable of generating convincing text, even from small datasets, and highlight the inadequacy of current detection and mitigation strategies.

Datasets

ENRON email dataset, various publicly available text samples from unspecified individuals.

Model(s)

Llama, Phi, GPT series, ChatGPT, Claude, Gemini, various open-source LLMs.

Author countries

Austria

← Previous