Transfer learning of a convolutional neural network for HEp-2 cell image classification

Phan, Ha Tran Hong; Kumar, Ashnil; Kim, Jin Man; Feng, Dagan

doi:10.1109/isbi.2016.7493483

Cited by 60 publications

(44 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many works add features derived from deep networks to existing feature sets or compare (2013) Detection of basal cell carcinoma H&E Convolutional auto-encoder neural network Malon and Cosatto (2013) Mitosis detection H&E Combines shapebased features with CNN Wang et al (2014) Mitosis detection H&E Cascaded ensemble of CNN and handcrafted features Ferrari et al (2015) Bacterial colony counting Culture plate CNN-based patch classifier Ronneberger et al (2015) Cell segmentation EM U-Net with deformation augmentation Shkolyar et al (2015) Mitosis detection Live-imaging CNN-based patch classifier Song et al (2015) Segmentation of cytoplasm and nuclei H&E Multi-scale CNN and graph-partitioning-based method Xie et al (2015a) Nucleus detection Ki-67 CNN model that learns the voting offset vectors and voting confidence Xie et al (2015b) Nucleus detection H&E, Ki-67 CNN-based structured regression model for cell detection Akram et al (2016) Cell segmentation FL, PC, H&E fCNN for cell bounding box proposal and CNN for segmentation Albarqouni et al (2016) Mitosis detection H&E Incorporated 'crowd sourcing' layer into the CNN framework Bauer et al (2016) Nucleus classification IHC CNN-based patch classifier Chen et al (2016b) Mitosis detection H&E Deep regression network (DRN) Gao et al (2016e) Nucleus classification IFL Classification of Hep2-cells with CNN Han et al (2016) Nucleus classification IFL Classification of Hep2-cells with CNN Janowczyk et al (2016b) Nucleus segmentation H&E Resolution adaptive deep hierarchical learning scheme Kashif et al (2016) Nucleus detection H&E Combination of CNN and hand-crafted features Mao and Yin (2016) Mitosis detection PC Hierarchical CNNs for patch sequence classification Mishra et al (2016) Classification of mitochondria EM CNN-based patch classifier Phan et al (2016) Nucleus classification FL Classification of Hep2-cells using transfer learning (pre-trained CNN) Romo-Bucheli et al (2016) Tubule nuclei detection H&E CNN-based classification of pre-selected candidate nuclei Sirinukunwattana et al (2016) Nucleus detection and classification H&E CNN with spatially constrained regression Song et al (2017) Cell segmentation H&E Multi-scale C...…”

Section: Chestmentioning

confidence: 99%

A survey on deep learning in medical image analysis

Litjens

Kooi

Bejnordi

et al. 2017

Medical Image Analysis

9,665

5,414

View full text Add to dashboard Cite

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks. Concise overviews are provided of studies per application area: neuro, retinal, pulmonary, digital pathology, breast, cardiac, abdominal, musculoskeletal. We end with a summary of the current state-of-the-art, a critical discussion of open challenges and directions for future research.

show abstract

Section: Chestmentioning

confidence: 99%

A survey on deep learning in medical image analysis

Litjens

Kooi

Bejnordi

et al. 2017

Medical Image Analysis

9,665

5,414

View full text Add to dashboard Cite

show abstract

“…Following the leave-one-donor-out test principle [31,38], we wanted the selection of the optimal hyperparameters to be generalizable to new donors as well. Therefore, we applied a nested cross-validation scheme [55,55,56] (Figure 8).…”

Section: Nested Cross-validationmentioning

confidence: 99%

Classifying T cell activity in autofluorescence intensity images with convolutional neural networks

Wang

Walsh

Skala

et al. 2019

Journal of Biophotonics

View full text Add to dashboard Cite

The importance of T cells in immunotherapy has motivated developing technologies to improve therapeutic efficacy. One objective is assessing antigen‐induced T cell activation because only functionally active T cells are capable of killing the desired targets. Autofluorescence imaging can distinguish T cell activity states in a non‐destructive manner by detecting endogenous changes in metabolic co‐enzymes such as NAD(P)H. However, recognizing robust activity patterns is computationally challenging in the absence of exogenous labels. We demonstrate machine learning methods that can accurately classify T cell activity across human donors from NAD(P)H intensity images. Using 8260 cropped single‐cell images from six donors, we evaluate classifiers ranging from traditional models that use previously‐extracted image features to convolutional neural networks (CNNs) pre‐trained on general non‐biological images. Adapting pre‐trained CNNs for the T cell activity classification task provides substantially better performance than traditional models or a simple CNN trained with the autofluorescence images alone. Visualizing the images with dimension reduction provides intuition into why the CNNs achieve higher accuracy than other approaches. Our image processing and classifier training software is available at https://github.com/gitter-lab/t-cell-classification.

show abstract

“…A trained model must be able to generalize to new donors in order to be useful in a practical pre-clinical or clinical setting. Therefore, all of our evaluation strategies train on images from some donors and evaluate the trained models on separate images from a different donor, which is referred to as subject-wise cross-validation 34 or a leave-one-patient-out scheme 28 . We initially assess the classifiers with cross-validation across donors.…”

Section: /29mentioning

confidence: 99%

“…Following the leave-one-donor-out test principle 28,34 , we wanted the selection of the optimal hyper-parameters to be generalizable to new donors as well. Therefore, we applied a nested cross-validation scheme 51,52 (Fig.…”

Section: Nested Cross-validationmentioning

confidence: 99%

Classifying T cell activity in autofluorescence intensity images with convolutional neural networks

Wang

Walsh

Skala

et al. 2019

Preprint

View full text Add to dashboard Cite

The importance of T cells in immunotherapy has motivated developing technologies to better characterize T cells and improve therapeutic efficacy. One specific objective is assessing antigen-induced T cell activation because only functionally active T cells are capable of killing the desired targets. Autofluorescence imaging can distinguish T cell activity states of individual cells in a non-destructive manner by detecting endogenous changes in metabolic co-enzymes such as NAD(P)H. However, recognizing robust patterns of T cell activity is computationally challenging in the absence of exogenous labels or information-rich autofluorescence lifetime measurements. We demonstrate that advanced machine learning can accurately classify T cell activity from NAD(P)H intensity images and that those image-based signatures transfer across human donors. Using a dataset of 8,260 cropped single-cell images from six donors, we meticulously evaluate multiple machine learning models. These range from traditional models that represent images using summary statistics or extract image features with CellProfiler to deep convolutional neural networks (CNNs) pre-trained on general non-biological images. Adapting pre-trained CNNs for the T cell activity classification task provides substantially better performance than traditional models or a simple CNN trained with the autofluorescence images alone. Visualizing the images with dimension reduction provides intuition into why the CNNs achieve higher accuracy than other approaches. However, we observe that fine-tuning all layers of the pre-trained CNN does not provide a classification performance boost commensurate with the additional computational cost. Our software detailing our image processing and model training pipeline is available as Jupyter notebooks at https: //github.com/gitter-lab/t-cell-classification. lifetime imaging to identify macrophages within the tumor microenvironment in vivo 7 and classified activation state of T cells in vitro 6 . The fluorescence lifetime of NAD(P)H and FAD is highly sensitive to the microenvironment and binding of NAD(P)H and FAD. Thus, this fluorescence lifetime can be used to resolve metabolic differences between functional states of immune cells. However, fluorescence lifetime imaging requires specialized and expensive microscope components, limiting its use to particular labs. Although autofluorescence intensity images lack the depth of information provided by the fluorescence lifetime, intensity images can easily and quickly be acquired on almost any commercial fluorescence microscope, allowing widespread adoption and seamless integration of a new technique into existing protocols of live cell assessment. Here, we develop a computational framework that uses autofluorescence intensity images to assess T cell activation state at the single-cell level.Machine learning is promising for classifying cell subtypes from label-free images. For example, Pavillon et al. used regularized logistic regression to predict macrophage activation state 8 , and Yoo...

show abstract

Transfer learning of a convolutional neural network for HEp-2 cell image classification

Cited by 60 publications

References 8 publications

A survey on deep learning in medical image analysis

A survey on deep learning in medical image analysis

Classifying T cell activity in autofluorescence intensity images with convolutional neural networks

Classifying T cell activity in autofluorescence intensity images with convolutional neural networks

Contact Info

Product

Resources

About