Heterogeneous Auto-similarities of Characteristics (HASC): Exploiting Relational Information for Classification

Abstract

Capturing the essential characteristics of visual objects by considering how their features are inter-related is a recent philosophy of object classification. In this paper, we embed this principle in a novel image descriptor, dubbed Heterogeneous Auto-Similarities of Characteristics (HASC). HASC is applied to heterogeneous dense features maps, encoding linear relations by covariances and nonlinear associations through information-theoretic measures such as mutual information and entropy. In this way, highly complex structural information can be expressed in a compact, scale invariant and robust manner. The effectiveness of HASC is tested on many diverse detection and classification scenarios, considering objects, textures and pedestrians, on widely known benchmarks Brodatz,

Experiments of Image Classification Using Dissimilarity Spaces Built with Siamese Networks

“…The feature vector is obtained by calculating the pattern's distance to the set of prototypes, P. This feature vector is fed into the SVM to determine its class. Both the original images in the data sets and the HASC [42] descriptors (outlined in Section 2.5) serve as the input to the classification process.…”

“…HASC [ 42 ] is a local descriptor designed to capture the linear covariances (COV) and nonlinear entropy combined with mutual information (EMI) relational characteristics of an object. Some of the advantages of covariance matrices as descriptors include their low dimension, robustness to noise, and their ability to capture the features of the joint PDF.…”

“…To accomplish this goal, the new system is built with a large set of eight different CNN architectures selected for the twin classifiers, with four new CNN architectures presented here. Heterogeneous auto-similarities of characteristics (HASC) [ 42 ] features are extracted from the aforementioned bird [ 39 ] and cat [ 40 , 41 ] data sets as well as on a medical data set for classifying narrow-band imaging (NBI) endoscopic videos [ 43 ] and a data set of images for the classification of the maturation of human stem cell-derived retinal pigmented epithelium [ 44 ]. In the training phase, a clustering algorithm is employed to select a set of relevant samples to be used as the prototypes of the training samples.…”

Bird and whale species identification using sound images

“…• RICLBP [29], a multi-scale rotation invariant co-occurrence of adjacent LBP with values (R = 1, P = 8), (R = 2, P = 8) and (R = 4, P = 8). • MLPQ [30] [31] are applied to heterogeneous dense features maps. • BSIF [23], where the standard BSIF descriptor is extracted by projecting sub-windows of the entire image onto sub-spaces.…”

“…Each LPQ trains a different classifier whose results are combined by sum rule. This ensemble is built up with 105 descriptors, and the scores are summed and normalised by dividing the sum by 105. Heterogeneous auto‐similarities of characteristics (HASC) [31] are applied to heterogeneous dense features maps. BSIF [23], where the standard BSIF descriptor is extracted by projecting sub‐windows of the entire image onto sub‐spaces. The images are binarised (using a threshold th ) and a histogram is built (see [32] for details).…”