Metadata-Guided Visual Representation Learning for Biomedical Images

Spiegel, Stephan; Hossain, Imtiaz; Ball, Christopher; Zhang, Xian

doi:10.1101/725754

Cited by 8 publications

(15 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Domain-specific representation learning and transfer learning are active research topics in the biomedical imaging field. 12 –15…”

Section: Introductionmentioning

confidence: 99%

“…Domain-specific representation learning and transfer learning are active research topics in the biomedical imaging field. [12][13][14][15] Since the collection and annotation of digital tissue slides are time-consuming and cumbersome, one looks for other imaging domains, where the image data and corresponding annotations are abundant and thus one could try to use models trained on such data for transfer learning. One large publicly available data set is ImageNet, which consists of more than 14 million images labelled with over 21,000 classes that are often used for benchmarking visual object recognition and image classification.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Biomarker-Based Classification and Localization of Renal Lesions Using Learned Representations of Histology—A Machine Learning Approach to Histopathology

et al. 2021

Self Cite

View full text Add to dashboard Cite

Several deep learning approaches have been proposed to address the challenges in computational pathology by learning structural details in an unbiased way. Transfer learning allows starting from a learned representation of a pretrained model to be directly used or fine-tuned for a new domain. However, in histopathology, the problem domain is tissue-specific and putting together a labelled data set is challenging. On the other hand, whole slide-level annotations, such as biomarker levels, are much easier to obtain. We compare two pretrained models, one histology-specific and one from ImageNet on various computational pathology tasks. We show that a domain-specific model (HistoNet) contains richer information for biomarker classification, localization of biomarker-relevant morphology within a slide, and the prediction of expert-graded features. We use a weakly supervised approach to discriminate slides based on biomarker level and simultaneously predict which regions contribute to that prediction. We employ multitask learning to show that learned representations correlate with morphological features graded by expert pathologists. All of these results are demonstrated in the context of renal toxicity in a mechanistic study of compound toxicity in rat models. Our results emphasize the importance of histology-specific models and their knowledge representations for solving a wide range of computational pathology tasks.

show abstract

“…Domain-specific representation learning and transfer learning are active research topics in the biomedical imaging field. 12 –15…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Biomarker-Based Classification and Localization of Renal Lesions Using Learned Representations of Histology—A Machine Learning Approach to Histopathology

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…Alternatively, [3] proposes a weakly supervised learning method. Other works as in [12,28] use the metadata information to devise pseudo-labels for the network supervision. All the fully or weakly supervised methods above suffer from the requirement of labeled sets, and annotating such images by experts leads to high costs and time-consuming efforts.…”

Section: Introductionmentioning

confidence: 99%

“…All the fully or weakly supervised methods above suffer from the requirement of labeled sets, and annotating such images by experts leads to high costs and time-consuming efforts. Even in [12,28], the labels acquired from metadata can be prone to imprecise labeling due to the nature of the data and treatment analysis techniques. Some other works use transfer learning techniques [1,26].…”

Section: Introductionmentioning

confidence: 99%

Contrastive Learning of Single-Cell Phenotypic Representations for Treatment Classification

Perakis,

Gorji,

Jain

et al. 2021

Preprint

View full text Add to dashboard Cite

Learning robust representations to discriminate cell phenotypes based on microscopy images is important for drug discovery. Drug development efforts typically analyse thousands of cell images to screen for potential treatments. Early works focus on creating hand-engineered features from these images or learn such features with deep neural networks in a fully or weakly-supervised framework. Both require prior knowledge or labelled datasets. Therefore, subsequent works propose unsupervised approaches based on generative models to learn these representations. Recently, representations learned with self-supervised contrastive lossbased methods have yielded state-of-the-art results on various imaging tasks compared to earlier unsupervised approaches. In this work, we leverage a contrastive learning framework to learn appropriate representations from single-cell fluorescent microscopy images for the task of Mechanism-of-Action classification. The proposed work is evaluated on the annotated BBBC021 dataset, and we obtain state-of-the-art results in NSC, NCSB and drop metrics for an unsupervised approach. We observe an improvement of 10% in NCSB accuracy and 11% in NSC-NSCB drop over the previously best unsupervised method. Moreover, the performance of our unsupervised approach ties with the best supervised approach. Additionally, we observe that our framework performs well even without post-processing, unlike earlier methods. With this, we conclude that one can learn robust cell representations with contrastive learning.

show abstract

“…In addition, batch effects could cause images with identical metadata to look different. All the methods above, besides Ljosa et al, (2013), use deep neural networks trained either in a supervised (Kraus et al, 2016;Godinez et al, 2017), self-supervised setup (Godinez et al, 2018;Spiegel et al, 2019) on an annotated subset of the data or make use of a neural network pre-trained on non-cellular images (Ando et al, 2017;Tabak et al, 2019). Using a neural network trained on non-cellular images yields the risk of domain-specific features being lost in the process of capturing features on data not related to HCS.…”

Section: Introductionmentioning

confidence: 99%

Fully unsupervised deep mode of action learning for phenotyping high-content cellular images

Janssens

Zhou

Kauffmann

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

The identification and discovery of phenotypes from high content screening (HCS) images is a challenging task. Earlier works use image analysis pipelines to extract biological features, supervised training methods or generate features with neural networks pretrained on non-cellular images. We introduce a novel fully unsupervised deep learning algorithm to cluster cellular images with similar Mode-of-Action together using only the images’ pixel intensity values as input. The method outperforms existing approaches on the labelled subset of the BBBC021 dataset and achieves an accuracy of 97.09% for correctly classifying the Mode-of-Action (MOA) by nearest neighbors matching. One unique aspect of the approach is that it is able to perform training on the entire unannotated dataset, to correctly cluster similar treatments beyond the annotated subset of the dataset and can be used for novel MOA discovery.

show abstract

Metadata-Guided Visual Representation Learning for Biomedical Images

Cited by 8 publications

References 21 publications

Biomarker-Based Classification and Localization of Renal Lesions Using Learned Representations of Histology—A Machine Learning Approach to Histopathology

Biomarker-Based Classification and Localization of Renal Lesions Using Learned Representations of Histology—A Machine Learning Approach to Histopathology

Contrastive Learning of Single-Cell Phenotypic Representations for Treatment Classification

Fully unsupervised deep mode of action learning for phenotyping high-content cellular images

Contact Info

Product

Resources

About