PlexNet: An Ensemble of Deep Neural Networks for Biometric Template Protection

Abstract

The security of biometric systems, especially protecting the templates stored in the gallery database, is a primary concern for researchers. This paper presents a novel framework using an ensemble of deep neural networks to protect biometric features stored as a template. The proposed ensemble chooses two state-of-the-art CNN architectures i.e., ResNet and DenseNet as base models for training. While training, the pre-trained weights enable the learning algorithm to converge faster. The weights obtained through the base model is further used to train other compatible models, generating a fine-tuned model. Thus, four fine-tuned models are prepared, and their learning are fused to form an ensemble named as PlexNet. To analyze biometric templates' security, the rigorous learning of ensemble is collected using a smart box i.e., application programming interface (API). The API is robust and correctly identifies the query image without referring to a template database. Thus, the proposed framework excludes the templates from database and performed predictions based on learning that is irrevocable.

Biometric Template Protection for Neural-Network-based Face Recognition Systems: A Survey of Methods and Evaluation Techniques

“…Caltech Faces 1999 • Georgia Tech • 10k US Adult Faces [31] -MOBIO NN-learned [37], [38] CMU PIE • Extended Yale Face Database B • CMU Multi-PIE [39] CMU PIE • Extended Yale Face Database B [40] CMU PIE • FEI • Color FERET [41] CMU PIE • FEI • Color FERET [42] CMU PIE • FEI • Color FERET [43] MS-Celeb-1M • CASIA-WebFace • AT&T / ORL • Own CASIA-WebFace • AT&T / ORL • Own [44] LFW [45] CMU PIE • Extended Yale Face Database B • CMU Multi-PIE • WVU Multimodal [46] VGGFace2 • MegaFace [47] YouTube Faces • FaceScrub [48] VGGFace2…”

“…[12] Rank-1 IR (%) DET (FNIR vs. FPIR) (%) FNIR @ FPIR = {0, 0.1, 1} (%) [13] ROC (FRR vs. FAR, plus EER) (%) [14] TAR @ FAR = {1, 0.1, 0.01, 0.001} (%) ROC (TAR vs. FAR) [15] [16] {TP, FP, TN, FN} @ FPR = 0.001 [17] FNMR and FMR (%) ≈ [18] EER DET (Miss vs. False Alarm prob.) (%) ≈ [19] Accuracy (%) [20] EER (%) [21] Recognition rate (%) [22] Accuracy (%) DET (FRR vs. FAR, plus EER) (%) [23] AUC (%), EER (%), TPR (%), TAR @ FAR = 0.1 (%), Rank-1 DIR @ FAR = 1 (%) [24] EER (%), GAR @ FAR (%) [25] FAR and FRR (%) [26] EER (%) [27] EER ROC (1 -FNMR vs. FMR) [28] TAR @ FAR (%) [29] IR (%) @ Rank = {1, 10, 50}, ROC (VR vs. FAR) (%) ≈ Accuracy (%), Verification Recognition @ FAR = 0.1 (%), Rank-1 DIR @ FAR = 1 (%), Own metrics: TIR, MIR, FIR (%) [30] Accuracy, FAR, FRR, EER (%) [31] 1 -FNMR (TMR) @ FMR = 10 −3 (or 0.1%) ROC (1 -FNMR vs. FMR) NN-learned [37], [38] EER (%), GAR @ FAR = {0, 1} (%) [39] EER (%), GAR @ FAR = 1 (%) ROC (GAR vs. FAR) [40] EER (%), GAR @ FAR = {0, 0.01, 0.1} (%) ROC (GAR vs. FAR) [41] EER (%), GAR @ FAR = 0 (%) [42] EER (%), GAR @ FAR (%) [43] FAR and FRR (%) [44] EER [45] EER (%), GAR @ FAR = 0.01 (%) ROC (GAR vs. FAR) (%) [46] Accuracy (%), GAR @ FAR = 0 (%) ROC (TAR vs. FAR) [47] EER (%), GAR @ FAR = 0.1 (%), Maximum Average Precision [48] GAR @ FAR = 0.1 (%) [49] EER (%), FNMR @ FMR = 0.1 (%) DET (FNMR vs. FMR) [50] EER (%) [51] EER (%) ROC (TPR vs. FPR) of the "other" systems (i.e., those using unprotected templates or templates protected by other BTP methods) were deliber-ately/directly compared (e.g., in a table, on the same plot, or conceptually). On the other hand, an implicit/pa...…”

“…Of the NN-learned BTP works that provided an explicit recognition accuracy comparison to unprotected templates, [46], [49]- [51] performed this evaluation by directly accessing these templates. In particular, the unprotected face templates came from the individual (pre-fusion) NNs in [46], the NN trained using the original Triplet Loss instead of the proposed Secure Triplet Loss formulation in [49], and straight from the pre-trained DNN that was used as the feature extractor preceding the BTP NN in [50], [51]. Alternatively, for the endto-end trained BTP NN in [48], the comparison was based on unprotected templates that were extracted using other "stateof-the-art" feature extractors.…”

“…The issue of not allowing for template renewability in the event of compromise, would also be present in [46], which proposes a face BTP method based on feature fusion. Four different NNs are trained to extract face features from the input face image, and the four sets of learned face features are fused via concatenation.…”

“…output) [30] FaceNet [83] https://bit.ly/3BFc2Fm (model: Inception-ResNet-v1, 512-dim. output) [31] FaceNet https://bit.ly/3nv3FYV Idiap https://bit.ly/3kaLTYQ NN-learned [37], [38] --- [39] CaffeNet [97] https://bit.ly/3BELMLm [40] VGG-Face [91] https://bit.ly/3tpmGgF [41] VGG-Face [91] https://bit.ly/3tpmGgF [42] VGG-Face [91] https://bit.ly/3tpmGgF [43] Inception-ResNet-v1 [98] https://bit.ly/3kNNCCJ [44] ResNet-11 [45] VGG-19 [99] https://bit.ly/2Yt3x1X [46] ResNet [89] DenseNet [100] https://bit.ly/38DAFG4 [47] VGG-16 [99] https://bit.ly/2Yt3x1X [48] ResNet-50 [ Table VII summarises the metrics and plots used to evaluate the recognition accuracy of the studied face BTP methods. Note that other metrics/plots may have been adopted for some evaluations, but Table VII presents only those used to report the final recognition accuracy (not those used in intermediate processing stages, like for fine-tuning system parameters).…”