SeeABLE: Soft Discrepancies and Bounded Contrastive Learning for Exposing Deepfakes

View on arXiv ← Back to list

Authors: Nicolas Larue, Ngoc-Son Vu, Vitomir Struc, Peter Peer, Vassilis Christophides

Published: 2022-11-21 09:38:30+00:00

AI Summary

SeeABLE is a novel deepfake detector that frames the problem as a one-class out-of-distribution detection task, improving generalization to unseen deepfakes. It generates local image perturbations (soft discrepancies) and uses a bounded contrastive loss to push perturbed faces towards prototypes, enhancing generalization by localizing modifications and identifying alteration types.

Abstract

Modern deepfake detectors have achieved encouraging results, when training and test images are drawn from the same data collection. However, when these detectors are applied to images produced with unknown deepfake-generation techniques, considerable performance degradations are commonly observed. In this paper, we propose a novel deepfake detector, called SeeABLE, that formalizes the detection problem as a (one-class) out-of-distribution detection task and generalizes better to unseen deepfakes. Specifically, SeeABLE first generates local image perturbations (referred to as soft-discrepancies) and then pushes the perturbed faces towards predefined prototypes using a novel regression-based bounded contrastive loss. To strengthen the generalization performance of SeeABLE to unknown deepfake types, we generate a rich set of soft discrepancies and train the detector: (i) to localize, which part of the face was modified, and (ii) to identify the alteration type. To demonstrate the capabilities of SeeABLE, we perform rigorous experiments on several widely-used deepfake datasets and show that our model convincingly outperforms competing state-of-the-art detectors, while exhibiting highly encouraging generalization capabilities.

Key findings

SeeABLE outperforms state-of-the-art deepfake detectors in cross-dataset and cross-manipulation experiments. Its one-class self-supervised anomaly detection approach demonstrates superior generalization capabilities to unseen deepfake generation techniques. The model effectively uses localized soft discrepancies and a novel loss function to achieve these results.

Approach

SeeABLE generates subtle local image perturbations (soft discrepancies) on real face images. A bounded contrastive regression loss trains a model to map these perturbations to predefined prototypes, enabling detection based on the distance to these prototypes. This approach leverages localization and alteration type information for improved generalization.

Datasets

FF++, CDF-v2, DFDC-p, DFDC

Model(s)

A multi-task regressor trained with a novel bounded contrastive regression loss.

Author countries

France, Slovenia

← Previous