DeepFaceLab: Integrated, flexible and extensible face-swapping framework

View on arXiv ← Back to list

Authors: Ivan Perov, Daiheng Gao, Nikolay Chervoniy, Kunlin Liu, Sugasa Marangonda, Chris Umé, Dpfks, Carl Shift Facenheim, Luis RP, Jian Jiang, Sheng Zhang, Pingyu Wu, Bo Zhou, Weiming Zhang

Published: 2020-05-12 03:26:55+00:00

AI Summary

DeepFaceLab is introduced as a dominant open-source deepfake framework for high-quality face-swapping. It offers an integrated, flexible, and extensible pipeline, simplifying the process and enabling cinema-quality results.

Abstract

Deepfake defense not only requires the research of detection but also requires the efforts of generation methods. However, current deepfake methods suffer the effects of obscure workflow and poor performance. To solve this problem, we present DeepFaceLab, the current dominant deepfake framework for face-swapping. It provides the necessary tools as well as an easy-to-use way to conduct high-quality face-swapping. It also offers a flexible and loose coupling structure for people who need to strengthen their pipeline with other features without writing complicated boilerplate code. We detail the principles that drive the implementation of DeepFaceLab and introduce its pipeline, through which every aspect of the pipeline can be modified painlessly by users to achieve their customization purpose. It is noteworthy that DeepFaceLab could achieve cinema-quality results with high fidelity. We demonstrate the advantage of our system by comparing our approach with other face-swapping methods.For more information, please visit:https://github.com/iperov/DeepFaceLab/.

Key findings

DeepFaceLab demonstrates competitive performance compared to other methods in terms of pose and expression retention and image quality. Ablation studies show the effectiveness of different model architectures, training paradigms, and the addition of GANs and TrueFace for improving results. The framework's flexibility and ease of use contribute to its popularity.

Approach

DeepFaceLab uses a three-phase pipeline: extraction (face detection, alignment, segmentation), training (using DF and LIAE structures with various loss functions and optimization techniques), and conversion (blending, color transfer, and sharpening). It supports large datasets and offers various customization options.

Datasets

FaceForensics++

Model(s)

S3FD (face detection), 2DFAN and PRNet (facial landmark extraction), TernausNet (face segmentation), DF and LIAE (face-swapping models). Optional GANs and TrueFace are also used.

Author countries

China, USA

← Previous