Heterogeneity-Aware Local Binary Patterns for Retrieval of Histopathology Images

Abstract

Histopathology images exhibit considerable variability, which can make diagnosis prone to uncertainty and errors. Using retrieval systems to locate similar images when a query image is given can assist pathologists in making more reliable decisions when diagnosing, based on accurately diagnosed past cases. Local binary patterns (LBP) have been successfully used as image descriptors for different applications. However, using LBP on histopathology images is still under investigation from different perspectives. The immense texture variability of these images and the lack of labeled training data are among the challenges that must be addressed. In this paper, we propose a new extension of LBP that is explicitly aware of the heterogeneity of local texture patterns through heterogeneity-based weighting. We use both homogeneity and the second moment (variance) of local neighborhoods to extend LBP histograms with heterogeneity information so that they better capture the polymorphism in histopathology images. We applied all concepts at multiple scales and investigated the extensions through both separated and concatenated extended histograms. We applied the proposed method on three publicly available datasets, namely, KimiaPath24, invasive ductal carcinoma (IDC), and BreakHis. The experimental results revealed that our features could retrieve and classify images with the highest accuracy. For KimiaPath24, we achieved 96.4% surpassing both LBP (91.0%) and deep features from VGG network (79.5%). For IDC, we reached the highest F1 measure (0.7665). Only for BreakHis, the proposed method was slightly less accurate than a customized convolutional neural network with rule fusion.INDEX TERMS Content based image retrieval (CBIR), whole slide image (WSI) processing, heterogeneity mask, local binary pattern (LBP).

Histopathology WSI Encoding based on GCNs for Scalable and Efficient Retrieval of Diagnostically Relevant Regions

“…A number of works concentrated on extracting high-level features of histopathological images to improve the accuracy of retrieval. Specifically, CBHIR frameworks based on manifold learning [11,37], semantic analysis [4, 5, 55], spectral embedding [38] and fine-designed local descriptors [39,12] have been developed and have proven effective in improving the accuracy of retrieval. Meanwhile, other works [15,57,16] proposed utilizing the contextual information by combining features from multiple magnifications of histopathological images to enhance the representations of image patches and thus improve the performance of retrieval.…”

Advanced deep learning applications in diagnostic pathology

“…Representations of two images should be closely related when the content of the images is similar and vice versa. Previously, many image representations have been used for CBIR in the field of diagnostic pathology including local binary patterns or their variants [13] and scale invariant feature transform [14]. Additionally, some methods have used unsupervised or self-supervised machine learning models to optimize image features [15] [16] (preprint).…”

Breast Cancer Classification from Histopathological Images Using Transfer Learning and Deep Neural Networks

“…They implement a system that reduces the workload of pathologists as well as improve the quality of diagnosis. In paper [20], the authors proposed a new system based on local binary patterns histograms on histopathology images that explicitly aware of the heterogeneity of local texture patterns through heterogeneity-based weighting. They used homogeneity and the second moment (variance) of local neighbourhoods based on heterogeneity information, so that makes better capture in histopathology images.…”