Balancing The Perception of Cheating Detection, Privacy and Fairness: A Mixed-Methods Study of Visual Data Obfuscation in Remote Proctoring

View on arXiv ← Back to list

Authors: Suvadeep Mukherjee, Verena Distler, Gabriele Lenzini, Pedro Cardoso-Leite

Published: 2024-06-21 11:40:56+00:00

AI Summary

This mixed-methods study investigates the impact of selectively obfuscating video recordings in remote proctoring on perceptions of cheating detection, privacy, and fairness. It finds that tailored region-specific obfuscation methods can improve privacy and fairness perceptions but may decrease perceived cheating detection effectiveness; however, participants preferred conventional blurring for videos they were more willing to share.

Abstract

Remote proctoring technology, a cheating-preventive measure, often raises privacy and fairness concerns that may affect test-takers' experiences and the validity of test results. Our study explores how selectively obfuscating information in video recordings can protect test-takers' privacy while ensuring effective and fair cheating detection. Interviews with experts (N=9) identified four key video regions indicative of potential cheating behaviors: the test-taker's face, body, background and the presence of individuals in the background. Experts recommended specific obfuscation methods for each region based on privacy significance and cheating behavior frequency, ranging from conventional blurring to advanced methods like replacement with deepfake, 3D avatars and silhouetting. We then conducted a vignette experiment with potential test-takers (N=259, non-experts) to evaluate their perceptions of cheating detection, visual privacy and fairness, using descriptions and examples of still images for each expert-recommended combination of video regions and obfuscation methods. Our results indicate that the effectiveness of obfuscation methods varies by region. Tailoring remote proctoring with region-specific advanced obfuscation methods can improve the perceptions of privacy and fairness compared to the conventional methods, though it may decrease perceived information sufficiency for detecting cheating. However, non-experts preferred conventional blurring for videos they were more willing to share, highlighting a gap between the perceived effectiveness of the advanced obfuscation methods and their practical acceptance. This study contributes to the field of user-centered privacy by suggesting promising directions to address current remote proctoring challenges and guiding future research.

Key findings

Advanced obfuscation methods (deepfake, 3D avatars) improved perceptions of privacy and fairness compared to blurring, but potentially reduced perceived information sufficiency for cheating detection. Participants showed a preference for blurring despite the higher perceived effectiveness of advanced methods in other aspects.

Approach

The study used a two-part mixed-methods approach. First, expert interviews identified key video regions and suitable obfuscation methods. Second, a vignette experiment with non-experts evaluated the perceptions of these methods regarding cheating detection, privacy, fairness, and willingness to share.

Datasets

Simulated images of test-takers in various scenarios, manipulated to apply different obfuscation techniques (blurring, deepfake, 3D avatar, silhouette) to different regions (face, body, background, individuals in background). Data from 259 non-expert participants and 9 experts were collected.

Model(s)

UNKNOWN (The study focuses on the application of various image manipulation techniques like blurring, deepfake, and 3D avatar creation, not on specific deepfake detection models.)

Author countries

Luxembourg, Germany

← Previous