Deep Face Fuzzy Vault: Implementation and Performance

Abstract

Deep convolutional neural networks have achieved remarkable improvements in facial recognition performance. Similar kinds of developments, e.g. deconvolutional neural networks, have shown impressive results for reconstructing face images from their corresponding embeddings in the latent space. This poses a severe security risk which necessitates the protection of stored deep face embeddings in order to prevent from misuse, e.g. identity fraud.In this work, an unlinkable improved deep face fuzzy vaultbased template protection scheme is presented. To this end, a feature transformation method is introduced which maps fixedlength real-valued deep face embeddings to integer-valued feature sets. As part of said feature transformation, a detailed analysis of different feature quantisation and binarisation techniques is conducted using features extracted with a state-of-the-art deep convolutional neural network trained with the additive angular margin loss (ArcFace). At key binding, obtained feature sets are locked in an unlinkable improved fuzzy vault. For key retrieval, the efficiency of different polynomial reconstruction techniques is investigated. The proposed feature transformation method and template protection scheme are agnostic of the biometric characteristic and, thus, can be applied to virtually any biometric features computed by a deep neural network.For the best configuration, a false non-match rate below 1% at a false match rate of 0.01%, is achieved in cross-database experiments on the FERET and FRGCv2 face databases. On average, a security level of up to approximately 28 bits is obtained. This work presents the first effective face-based fuzzy vault scheme providing privacy protection of facial reference data as well as digital key derivation from face.

Towards Protecting Face Embeddings in Mobile Face Verification Scenarios

“…These methods use some sort of algorithmically defined transformation to convert the face embeddings (learned from the input face images) to more secure representations. The proposed transformations include one-way cryptographic [6] or Winner Takes All [7] hashing, convolution of the embedding with a random kernel [8], use of the Fuzzy Commitment [9] or Fuzzy Vault [10] scheme, fusion of a subject's face embedding with a different subject's face embedding using keys extracted from the two sets of features [11], and homomorphic encryption [12]. The main issue with these approaches is that they have not been comprehensively evaluated in terms of their ability to simultaneously satisfy all three properties of face embedding protection methods.…”

“…To analyse PolyProtect's irreversibility, we must first define our threat model, as specified in ISO/IEC 30316 (the international standard on performance testing of biometric template protection schemes) 10 . The threat model characterises the type of attacker on which we wish to base our irreversibility analysis.…”

“…So, the first step was to set up a system of k × p equations for each overlap amount 16 , and for each value of p in the range [1,10]. For example, for overlap = 0, to simulate an attacker with access to the minimum of only one 26-dimensional P , we set up a system of 26 equations in 128 unknowns (embedding elements), same as for the irreversibility analysis in Section 4.3.2.…”

“…Similarly to the analysis in Section 4.3.2, we evaluated the inversion success rate = solution rate × match rate. This was calculated separately for each p in the range [1,10] (i.e., 1 to 10 PolyProtected templates per V ), and for each overlap amount. Note that, since the aim of this analysis was to determine whether using multiple PolyProtected templates would make it easier to recover the original face embedding than using only one PolyProtected template, the choice of match threshold was not so important, particularly because the irreversibility analysis in Section 4.3.2 already demonstrated the effects of using different match thresholds.…”

Biometric Template Protection for Neural-Network-based Face Recognition Systems: A Survey of Methods and Evaluation Techniques

“…During authentication, the probe biometric template is used to determine the true points and thereby recover the secret polynomial from the reference fuzzy vault, which will be successful if the cryptographic hash of the recovered set of polynomial coefficients matches the hash of the set of reference coefficients. The Fuzzy Vault scheme was proposed as a Non-NN face BTP method in [17].…”