Cascaded Static and Dynamic Local Feature Extractions for Face Sketch to Photo Matching

Abstract

The automatic identification of a corresponding photo from a face sketch can assist in criminal investigations. The face sketch is rendered based on the descriptions elicited by the eyewitness. This may cause the face sketch to have some degrees of shape exaggeration that make some parts of the face geometrically misaligned. In this paper, we attempt to address the effect of these influences by a cascaded static and dynamic local feature extraction method so that the constructed feature vectors are built based on the correct patches. In the proposed method, the feature vectors from the local static extraction on a sketch and photo are matched using the nearest neighbors. Then, some n most similar photos are shortlisted based on the nearest neighbors. These photos are eventually re-matched using feature vectors from the local dynamic extraction method. The feature vectors are matched using the L 1 -distance measure. The experimental results for The Chinese University of Hong Kong (CUHK) Face Sketch Database (CUFS) and CUHK Face Sketch FERET Database (CUFSF) datasets indicate that the proposed method outperforms the state-of-the-art methods.INDEX TERMS Cascaded feature extraction, identity of interest, local feature extraction, sketch to photo, static and dynamic, deep learning.

Canonical Correlation Analysis Feature Fusion With Patch of Interest: A Dynamic Local Feature Matching for Face Sketch Image Retrieval

“…A common representation of the face sketch and photo is extracted and matched. It extracts discriminative features that are invariant to photo and sketch modalities before performing similarity measure [15]- [19], [26], [44]. Klare and Jain [14] proposed a local feature extraction approach using Scale Invariant Feature Transform (SIFT) descriptor.…”

“…This is because DoGOGH has been proven to perform effectively on matching face sketch to photo with illumination effects [26]. For the latter problem, we propose such that the local feature vector is extracted dynamically (i.e., based on a reference patch) using dynamic local feature extraction method [44]. This is to cater a possibility of the exaggerated shape resides in the neighbouring patches.…”

“…If the feature is extracted from a proper local position, the retrieval rate could be increased due to the fact that the feature matching is made on the true pairs. To address this, here, we use dynamic local feature extraction as described in [44]. In the context of matching sketch to photo, dynamic local feature extraction extracts feature vector from local patch (of the photo) dynamically based on a reference feature vector (extracted from sketch) at the same local position.…”

“…We call this method as static extraction method because the features are extracted out from patches at fixed local position. In addition to that, another local feature extraction method as in [44] is employed (i.e., D-DoGOGH) to extract dynamic feature from the photo. To dynamically extract the feature vector f P m from photo, let consider…”

“…Then, for the sketch, extract DoGOGH feature F S on these points (which later be the reference feature for dynamic extraction on photo). Similarly, for the photo, extract DoGOGH feature F P on these points and its neighbouring points as what has been proposed in [44]. Based on these feature vectors, the distances between f S m and f P ml are computed using L 1distance measure.…”