Labeling Synthetic Content: User Perceptions of Warning Label Designs for AI-generated Content on Social Media

View on arXiv ← Back to list

Authors: Dilrukshi Gamage, Dilki Sewwandi, Min Zhang, Arosha Bandara

Published: 2025-02-14 10:35:42+00:00

Comment: This is a pre print longer version of a paper accepted to CHI 2025; after rebuttal we had to short the paper to 25 pages. Currently its in overleaf manuscript format with one column. All data for the file is in the osf link

Journal Ref: CHI Conference on Human Factors in Computing Systems CHI 25, April 26-May 1, 2025, Yokohama, Japan

AI Summary

This research investigates the efficacy of various warning label designs for AI-generated content on social media platforms. Through an experimental study with 911 participants evaluating ten distinct label samples, the authors found that labels significantly increased users' belief that content was AI-generated, although trust in the labels varied based on design. Notably, the presence of labels did not significantly alter user engagement behaviors such as liking, commenting, or sharing.

Abstract

In this research, we explored the efficacy of various warning label designs for AI-generated content on social media platforms e.g., deepfakes. We devised and assessed ten distinct label design samples that varied across the dimensions of sentiment, color/iconography, positioning, and level of detail. Our experimental study involved 911 participants randomly assigned to these ten label designs and a control group evaluating social media content. We explored their perceptions relating to 1. Belief in the content being AI-generated, 2. Trust in the labels and 3. Social Media engagement perceptions of the content. The results demonstrate that the presence of labels had a significant effect on the users belief that the content is AI generated, deepfake, or edited by AI. However their trust in the label significantly varied based on the label design. Notably, having labels did not significantly change their engagement behaviors, such as like, comment, and sharing. However, there were significant differences in engagement based on content type: political and entertainment. This investigation contributes to the field of human computer interaction by defining a design space for label implementation and providing empirical support for the strategic use of labels to mitigate the risks associated with synthetically generated media.

Key findings
The presence of warning labels significantly increased users' belief that content was AI-generated, with trust varying significantly by label design; labels with 'Content Credentials' elicited the highest trust, which correlated with trust in the platform. However, labels did not significantly alter overall user engagement behaviors (like, comment, share), although engagement varied significantly based on content type (political vs. entertainment).
Approach
The researchers identified a design space for AI-generated content warning labels based on four dimensions: sentiment, iconography/color, positioning, and level of detail. They then created 10 prototype labels and conducted an experimental study with 911 participants to evaluate user perceptions of these labels on social media image posts, measuring belief in AI generation, trust in labels, and engagement behaviors.
Datasets
Eight images collected from various online sources, comprising four AI-generated/edited images and four real images, balanced between political and entertainment content.
Model(s)
UNKNOWN
Author countries
Sri Lanka, United Kingdom
← Previous
Next →