Photorealistic Face De-Identification by Aggregating Donors’ Face Components

Abstract

With the adoption of pervasive surveillance systems and the development of efficient automatic face matchers, the question of preserving privacy becomes paramount. In this context, automated face de-identification is revived. Typical solutions based on eyes masking or pixelization, while commonly used in news broadcasts, produce very unnatural images. More sophisticated solutions were sparingly introduced in the literature, but they fail to account for fundamental constraints such as the visual likeliness of de-identified images. In contrast, we identify essential principles and build upon efficient techniques to derive an automated face de-identification solution meeting our predefined criteria. More specifically, our approach relies on a set of face donors from which it can borrow various face components (eyes, chin, etc.). Faces are then de-identified by substituting their own face components with the donors' ones, in such a way that an automatic face matcher is fooled while the appearance of the generated faces are as close as possible to original faces. Experiments on several datasets validate the approach and show its ability both in terms of privacy preservation and visual quality.

On Face Segmentation, Face Swapping, and Face Perception

“…Methods for swapping faces were proposed as far back as 2004 by [5] with fully automatic techniques described nearly a decade ago in [4]. These methods were originally offered in response to privacy preservation concerns: Face swapping can be used to obfuscate identities of subjects appearing in publicly available photos, as a substitute to face pixelation or blurring [4,5,9,28,34,38]. Since then, however, many of their applications seem to come from recreation [19] or entertainment (e.g., [1,44]).…”

“…Swapping faces means transferring a face from a source photo onto a face appearing in a target photo, attempting to generate realistic, unedited looking results. Although face swapping today is often associated with viral Internet memes [10,35], it is actually far more important than this practice may suggest: Face swapping can also be used for preserving privacy [4,34,38], digital forensics [35] and as a potential face specific data augmentation method [33] especially in applications where training data is scarce (e.g., facial emotion recognition [26]).…”

FSGAN: Subject Agnostic Face Swapping and Reenactment

“…Methods for manipulating the appearances of face images, particularly for face swapping and reenactment, have a long history, going back nearly two decades. These methods were originally proposed due to privacy concerns [6,26,32] though they are increasingly used for recreation [21] or entertainment (e.g., [1,48]).…”

“…1 right). Both tasks are attracting significant research attention due to their applications in entertainment [1,21,48], privacy [6,26,32], and training data generation.…”

De-identifying facial images using singular value decomposition and projections

“…Other masks reveal only the mouth and, finally, a black mask can be applied to the entire face, destroying all visual information of the facial image [8]. Additional simple methods include methods that blur the face area using low-pass filters [8], methods that add random noise with a predetermined distribution, methods that use the negative image and methods that swap facial areas, such as eyes, nose, mouth, between images that belong to different individuals [11]. Finally, simple methods also exist that subsample an image leading to pixelation, or that threshold the pixel values [8].…”