Multi-task deep learning for fine-grained classification and grading in breast cancer histopathological images

Abstract

Fine-grained classification and grading of breast cancer (BC) histopathological images are of great value in clinical application. However, automatic classification and grading of BC histopathological images are complicated by (1) small inter-class variance and large intraclass variance exist in BC histopathological images, and (2) features extracted from similar histopathological images with different magnification are quite different. To address these issues, an improved deep convolution neural network model is proposed and the procedure can be divided into three main stages. Firstly, in the representation learning process, multiclass recognition task and verification task of image pair are combined. Secondly, in the feature extraction process, a prior knowledge is built, which is "the variances in feature outputs between different subclasses is relatively large while the variance between the same subclass is small." Additionally, the prior information that histopathological images with different magnification belong to the same subclass are embedded in the feature extraction process, which contributes to less sensitive with image magnification. The experimental results based on three different histopathological image datasets show that the performance of the proposed method is better than state of the art, with better robustness and generalization ability. Keywords Multi-task deep learning • Histopathological image classification • Fine-grained • Convolutional neural network • Breast cancer Lingqiao Li and Xipeng Pan are contributed equally

An Enhanced Histopathology Analysis: An AI-Based System for Multiclass Grading of Oral Squamous Cell Carcinoma and Segmenting of Epithelial and Stromal Tissue

“…However, histopathological classification of oral carcinoma can be challenging due to heterogeneous structure and textures, and the presence of any inflammatory tissue reaction. With the help of artificial intelligence-aided tools, the automatic classification of histopathological images can not only improve objective diagnostic results for the clinician but also provide detailed texture analysis in order to get an accurate diagnosis [ 58 ].…”

Fine-Grained Breast Cancer Classification With Bilinear Convolutional Neural Networks (BCNNs)

“…Most of the previously published breast cancer classification methods based on BreaKHis ( Cascianelli et al, 2017 ; Gupta and Bhavsar, 2017 , 2018 ; Han et al, 2017 ; Song et al, 2017 ; Wei et al, 2017 ; Bardou et al, 2018 ; Benhammou et al, 2018 ; Gandomkar et al, 2018 ; Karthiga and Narasimhan, 2018 ; Li et al, 2018 ; Zhang et al, 2018 ) use binary classification and not fine-grained classification. Furthermore, most binary classification and all fine-grained classification approaches are magnification-specific.…”

“…Fine-grained image classification differentiates between hard-to-distinguish or similar subclasses in plants ( Nilsback and Zisserman, 2008 ), animals ( Catherine et al, 2011 ), and models of vehicles ( Krause et al, 2013 ). Some approaches of histopathological image classification do not address the peculiarity of histopathological images and do not use specialized fine-grained classification methods ( Han et al, 2017 ; Bardou et al, 2018 ; Gandomkar et al, 2018 ; Li et al, 2018 ; Yan et al, 2019 ). Previously, fine-grained image classification of histopathological images was shown to perform better than ordinary CNNs.…”

“…Feature output variations were relatively large between different subclasses and small within the same subclass. The outputs from this study were embedded into the feature extraction process by Li et al (2018) . There are three specialized methods for fine-grained classification, namely fine-grained feature learning-based methods ( Lin et al, 2015 ), object part annotation-based methods ( Zhang N. et al, 2014 ), and visual attention-based methods ( Jianlong et al, 2017 ).…”

Spectral–Spatial Features Integrated Convolution Neural Network for Breast Cancer Classification

“…It was reported that polynomial kernels achieved higher accuracy in comparison with linear and RBF kernel. In [ 27 ], a fine-grained BreaKHis classification model was proposed using transfer learning approach with Xception model. The architecture was built to multi-task CNN and combined two loss function including Euclidean distance and loss function from the softmax layer to classify images.…”