Deepfake tweets automatic detection

View on arXiv ← Back to list

Authors: Adam Frej, Adrian Kaminski, Piotr Marciniak, Szymon Szmajdzinski, Soveatin Kuntur, Anna Wroblewska

Published: 2024-06-24 09:55:31+00:00

AI Summary

This research focuses on detecting deepfake tweets using natural language processing (NLP) techniques. It evaluates various machine learning models on the TweepFake dataset to identify effective strategies for recognizing AI-generated text and improving the reliability of online information.

Abstract

This study addresses the critical challenge of detecting DeepFake tweets by leveraging advanced natural language processing (NLP) techniques to distinguish between genuine and AI-generated texts. Given the increasing prevalence of misinformation, our research utilizes the TweepFake dataset to train and evaluate various machine learning models. The objective is to identify effective strategies for recognizing DeepFake content, thereby enhancing the integrity of digital communications. By developing reliable methods for detecting AI-generated misinformation, this work contributes to a more trustworthy online information environment.

Key findings

The RoBERTa model achieved the highest balanced accuracy (0.896) and F1 score (0.897) using raw data from the TweepFake dataset. The study also found that RNN-generated tweets are easier to detect than GPT2-generated deepfakes, highlighting the increasing sophistication of AI-generated content.

Approach

The study employs various text representation and preprocessing methods, evaluating machine learning, deep learning, and transformer models (including LightGBM, XGBoost, Random Forest, Logistic Regression, SVM, CNN, GRU, and xlm-roberta-base, distilbert-base-uncased, GPT-2) on the TweepFake dataset and GPT-2 generated texts. The impact of features like emoticons, mentions, and URLs on detection accuracy is also analyzed.

Datasets

TweepFake dataset and GPT-2 generated texts

Model(s)

LightGBM, XGBoost, Random Forest, Logistic Regression, SVM, CNN, GRU, CNN+GRU, xlm-roberta-base, distilbert-base-uncased, GPT-2

Author countries

Poland

← Previous