Classifying and segmenting microscopy images with deep multiple instance learning

Abstract

Motivation: High-content screening (HCS) technologies have enabled large scale imaging experiments for studying cell biology and for drug screening. These systems produce hundreds of thousands of microscopy images per day and their utility depends on automated image analysis. Recently, deep learning approaches that learn feature representations directly from pixel intensity values have dominated object recognition challenges. These tasks typically have a single centered object per image and existing models are not directly applicable to microscopy datasets. Here we develop an approach that combines deep convolutional neural networks (CNNs) with multiple instance learning (MIL) in order to classify and segment microscopy images using only whole image level annotations.Results: We introduce a new neural network architecture that uses MIL to simultaneously classify and segment microscopy images with populations of cells. We base our approach on the similarity between the aggregation function used in MIL and pooling layers used in CNNs. To facilitate aggregating across large numbers of instances in CNN feature maps we present the Noisy-AND pooling function, a new MIL operator that is robust to outliers. Combining CNNs with MIL enables training CNNs using whole microscopy images with image level labels. We show that training end-to-end MIL CNNs outperforms several previous methods on both mammalian and yeast datasets without requiring any segmentation steps.Availability and implementation: Torch7 implementation available upon request.Contact: oren.kraus@mail.utoronto.ca

Cheminformatics in Drug Discovery, an Industrial Perspective

“…Examples of architectures are feed‐forward network that can be used for multi‐task QSAR modelling . Convolutional networks that are used for image analysis . Recurrent neural networks are used for de novo molecular design, Auto‐encoder networks that can also be used for de novo molecular design ,…”

“…[24] Convolutional networks that are used for image analysis. [25] Recurrent neural networks are used for de novo molecular design [26,27] Auto-encoder networks that can also be used for de novo molecular design. [28,29] Besides the machine learning algorithm, it is also important to be able to estimate the confidence in a rigorous way of the prediction.…”

Avoiding Over-Detection: Towards Combined Object Detection and Counting

“…In [8], an FCN is used trained to classify histological images at a whole-image level. Although it is only trained with whole-image labels, it is still able to localise individual cells by deriving class probability maps from the final convolutional layers, in a manner inspired by [20] and [22].…”

Computational biology: deep learning

“…The authors further combined cell-level predictions into a single, highly accurate, protein classification. A team from Toronto demonstrated on the same unsegmented data that are possible to identify a localization label within a region and an image-level label with convolutional neural networks in a single step [61]. This has the advantage that only image-level labels are used, precluding the need to perform cell segmentation first.…”