Fusion of Hash-Based Hard and Soft Biometrics for Enhancing Face Image Database Search and Retrieval

Abstract

The utilization of digital picture search and retrieval has grown substantially in numerous fields for different purposes during the last decade, owing to the continuing advances in image processing and computer vision approaches. In multiple real-life applications, for example, social media, content-based face picture retrieval is a well-invested technique for large-scale databases, where there is a significant necessity for reliable retrieval capabilities enabling quick search in a vast number of pictures. Humans widely employ faces for recognizing and identifying people. Thus, face recognition through formal or personal pictures is increasingly used in various reallife applications, such as helping crime investigators retrieve matching images from face image databases to identify victims and criminals. However, such face image retrieval becomes more challenging in large-scale databases, where traditional vision-based face analysis requires ample additional storage space than the raw face images already occupied to store extracted lengthy feature vectors and takes much longer to process and match thousands of face images. This work mainly contributes to enhancing face image retrieval performance in large-scale databases using hash codes inferred by locality-sensitive hashing (LSH) for facial hard and soft biometrics as (Hard BioHash) and (Soft BioHash), respectively, to be used as a search input for retrieving the top-k matching faces. Moreover, we propose the multi-biometric score-level fusion of both face hard and soft BioHashes (Hard-Soft BioHash Fusion) for further augmented face image retrieval. The experimental outcomes applied on the Labeled Faces in the Wild (LFW) dataset and the related attributes dataset (LFW-attributes), demonstrate that the retrieval performance of the suggested fusion approach (Hard-Soft BioHash Fusion) significantly improved the retrieval performance compared to solely using Hard BioHash or Soft BioHash in isolation, where the suggested method provides an augmented accuracy of 87% when executed on 1000 specimens and 77% on 5743 samples. These results remarkably outperform the results of the Hard BioHash method by (50% on the 1000 samples and 30% on the 5743 samples), and the Soft BioHash method by (78% on the 1000 samples and 63% on the 5743 samples).