PE-MIU: A Training-Free Privacy-Enhancing Face Recognition Approach Based on Minimum Information Units

Abstract

Research on soft-biometrics showed that privacy-sensitive information can be deduced from biometric data. Utilizing biometric templates only, information about a persons gender, age, ethnicity, sexual orientation, and health state can be deduced. For many applications, these templates are expected to be used for recognition purposes only. Thus, extracting this information raises major privacy issues. Previous work proposed two kinds of learning-based solutions for this problem. The first ones provide strong privacy-enhancements, but limited to pre-defined attributes. The second ones achieve more comprehensive but weaker privacy-improvements. In this work, we propose a Privacy-Enhancing face recognition approach based on Minimum Information Units (PE-MIU). PE-MIU, as we demonstrate in this work, is a privacy-enhancement approach for face recognition templates that achieves strong privacy-improvements and is not limited to pre-defined attributes. We exploit the structural differences between face recognition and facial attribute estimation by creating templates in a mixed representation of minimal information units. These representations contain pattern of privacy-sensitive attributes in a highly randomized form. Therefore, the estimation of these attributes becomes hard for function creep attacks. During verification, these units of a probe template are assigned to the units of a reference template by solving an optimal best-matching problem. This allows our approach to maintain a high recognition ability. The experiments are conducted on three publicly available datasets and with five state-of-the-art approaches. Moreover, we conduct the experiments simulating an attacker that knows and adapts to the systems privacy mechanism. The experiments demonstrate that PE-MIU is able to suppress privacy-sensitive information to a significantly higher degree than previous work in all investigated scenarios. At the same time, our solution is able to achieve a verification performance close to that of the unmodified recognition system. Unlike previous works, our approach offers a strong and comprehensive privacy-enhancement without the need of training.

Privacy–Enhancing Face Biometrics: A Comprehensive Survey

“…• Inference-level techniques: Privacy enhancement may also be applied during the matching or classification stages in a biometric system, a.k.a, during inference. Here, some properties of the matching or classification procedure are commonly exploited to ensure that the data is only used for the intended purpose, see e.g., [22]. Thus, inference-level B-PETs typically modify the biometric template as well as the comparison/classification procedure used to derive a similarity/comparison score in the biometric system with the goal of privacy enhancement.…”

“…The presented scheme combines Eigenfaces with the Paillier [258] and Damgård, Geisler and Krøigaard (DGK) [259] cryptosystems and enables projecting facial images into an Eigen-space, comparing queries to templates in the database, and finding matching identities from the database. Because only a matching function is defined in the encrypted domain and the database is assumed to be private, this scheme allows for identity inference but [22] GD, ET ID Unsupervised Minimum information units RD M LFW [127], Adience [132], ColorFERET [146] Symbol explanation: GD -gender, A -age, ET -ethnicity, ID -identity, RD -reduction, RT -retention, M -machine.…”

“…Another inference-level technique was introduced recently by the same authors [22]. The proposed Privacy-Enhancing face recognition approach is based on Minimum Information Units (PE-MIU) and is completely unsupervised.…”

“…Techniques that follow this idea are related to the data minimization principle set forth in GDPR, which suggests that the amount of personal data that needs to be processed for a specific purpose must be kept minimal. Since different terminology is used in the literature to describe such techniques, we refer to them collectively as Biometric Privacy-Enhancing Techniques (B-PETs) in this survey [12], [19], [20], [21], [22].…”

On Soft-Biometric Information Stored in Biometric Face Embeddings

“…First, for many applications, the users do only permit to have access to the information related to recognition [32] and extracting additional information without a person's consent is considered a violation of their privacy [24]. This is known as soft-biometric privacy [32] and solutions are either build on image- [30], [31], [34] or embedding-level [5], [42], [45], [51]. Second, the attributes stored in biometric face embeddings can indicate biased performances related to these attributes that might result in unfair performance differences.…”