Fusion of Intra- and Inter-modality Algorithms for Face-Sketch Recognition

Abstract

Abstract. Identifying and apprehending suspects by matching sketches created from eyewitness and victim descriptions to mugshot photos is a slow process since law enforcement agencies lack automated methods to perform this task. This paper attempts to tackle this problem by combining Eigentransformation, a global intra-modality approach, with the Eigenpatches local intra-modality technique. These algorithms are then fused with an inter-modality method called Histogram of Averaged Orientation Gradients (HAOG). Simulation results reveal that the intraand inter-modality algorithms considered in this work provide complementary information since not only does fusion of the global and local intra-modality methods yield better performance than either of the algorithms individually, but fusion with the inter-modality approach yields further improvement to achieve retrieval rates of 94.05% at Rank-100 on 420 photo-sketch pairs. This performance is achieved at Rank-25 when filtering of the gallery using demographic information is carried out.

Forensic Face Photo-Sketch Recognition Using a Deep Learning-Based Architecture

“…using a linear combination of images, the Eigen-patches (EP) extension [15] performing synthesis at a local level, and the Bayesian framework in [16] that considers relationships among neighbouring patches for model construction. A more thorough review of FH algorithms may be found in [4], [13], [15]- [17].…”

Robust Face Sketch Recognition Using Locality Sensitive Histograms

“…Existing face sketch recognition methods can be classified into two categories: intra-modality and inter-modality approaches [2]- [4]. Intra-modality approaches deploy a preprocessing step that synthesizes pseudo sketch from input photo and perform face sketch recognition in the same modality [2], [3], [7].…”

“…Existing face sketch recognition methods can be classified into two categories: intra-modality and inter-modality approaches [2]- [4]. Intra-modality approaches deploy a preprocessing step that synthesizes pseudo sketch from input photo and perform face sketch recognition in the same modality [2], [3], [7]. In contrast, inter-modality approaches try to extract more discriminative and modality-invariant features (such as histogram of gradients (HOG) [1]) from input sketch and photo and match them [2]- [4].…”

“…Therefore, in general, inter-modality approaches are less robust or less accurate, thus has been less explored than intra-modality ones. In the intra-modality approaches, pseudo sketches can be synthesized by various effective algorithms [2], [7]. However, there is a risk that imperfect synthesis can lead to poor recognition, even worse than directly matching the original input photo and sketch.…”

“…However, there is a risk that imperfect synthesis can lead to poor recognition, even worse than directly matching the original input photo and sketch. Therefore, it is not recommended to use each approach alone and it is much more effective to combine two approaches [2]. A recent study attempted to combine the intra-modality and inter-modality approaches [7].…”