Digital pathology represents one of the major evolutions in modern medicine. Pathological examinations constitute the gold standard in many medical protocols, and also play a critical and legal role in the diagnosis process. In the conventional cancer diagnosis, pathologists analyze biopsies to make diagnostic and prognostic assessments, mainly based on the cell morphology and architecture distribution. Recently, computerized methods have been rapidly evolving in the area of digital pathology, with growing applications related to nuclei detection, segmentation, and classification. In cancer research, these approaches have played, and will continue to play a key (often bottleneck) role in minimizing human intervention, consolidating pertinent second opinions, and providing traceable clinical information. Pathological studies have been conducted for numerous cancer detection and grading applications, including brain, breast, cervix, lung, and prostate cancer grading. Our study presents, discusses, and extracts the major trends from an exhaustive overview of various nuclei detection, segmentation, feature computation, and classification techniques used in histopathology imagery, specifically in hematoxylin-eosin and immunohistochemical staining protocols. This study also enables us to measure the challenges that remain, in order to reach robust analysis of whole slide images, essential high content imaging with diagnostic biomarkers and prognosis support in digital pathology.

The computation of good image descriptors is key to the instance retrieval problem and has been the object of much recent interest from the multimedia research community. With deep learning becoming the dominant approach in computer vision, the use of representations extracted from Convolutional Neural Nets (CNNs) is quickly gaining ground on Fisher Vectors (FVs) as favoured state-of-the-art global image descriptors for image instance retrieval. While the good performance of CNNs for image classification are unambiguously recognised, which of the two has the upper hand in the image retrieval context is not entirely clear yet.In this work, we propose a comprehensive study that systematically evaluates FVs and CNNs for image retrieval. The first part compares the performances of FVs and CNNs on multiple publicly available data sets. We investigate a number of details specific to each method. For FVs, we compare sparse descriptors based on interest point detectors with dense single-scale and multi-scale variants. For CNNs, we focus on understanding the impact of depth, architecture and training data on retrieval results. Our study shows that no descriptor is systematically better than the other and that performance gains can usually be obtained by using both types together. The second part of the study focuses on the impact of geometrical transformations such as rotations and scale changes. FVs based on interest point detectors are intrinsically resilient to such transformations while CNNs do not have a built-in mechanism to ensure such invariance. We show that performance of CNNs can quickly degrade in presence of rotations while they are far less affected by changes in scale. We then propose a number of ways to incorporate the required invariances in the CNN pipeline.Overall, our work is intended as a reference guide offering practically useful and simply implementable guidelines to anyone looking for state-of-the-art global descriptors best suited to their specific image instance retrieval problem. * V. Chandrasekhar, J. Lin and O. Morère contributed equally to this work.

Histopathological examination is a powerful method for the prognosis of critical diseases. But, despite significant advances in high-speed and high-resolution scanning devices or in virtual exploration capabilities, the clinical analysis of Whole Slide Images (WSI) largely remains the work of human experts. We propose an innovative platform in which multi-scale computer vision algorithms perform fast analysis of a histopathological WSI. It relies on specific high and generic low resolution image analysis algorithms embedded in a multi-scale framework to rapidly identify the high power fields of interest used by the pathologist to assess a global grading. GPU technologies as well speed up the global time-efficiency of the system. In a sense, sparse coding and sampling is the keystone of our approach. In terms of validation, we are designing a computer-aided breast biopsy analysis application based on histopathology images and designed in collaboration with a pathology department. The current ground truth slides correspond to about 36,000 high magnification (40X) high power fields. The time processing to achieve automatic WSI analysis is on a par with the pathologist's performance (about ten minutes a WSI), which constitutes by itself a major contribution of the proposed methodology.

Image instance retrieval is the problem of retrieving images from a database which contain the same object. Convolutional Neural Network (CNN) based descriptors are becoming the dominant approach for generating global image descriptors for the instance retrieval problem. One major drawback of CNN-based global descriptors is that uncompressed deep neural network models require hundreds of megabytes of storage making them inconvenient to deploy in mobile applications or in custom hardware. In this work, we study the problem of neural network model compression focusing on the image instance retrieval task. We study quantization, coding, pruning and weight sharing techniques for reducing model size for the instance retrieval problem. We provide extensive experimental results on the trade-off between retrieval performance and model size for different types of networks on several data sets providing the most comprehensive study on this topic. We compress models to the order of a few MBs: two orders of magnitude smaller than the uncompressed models while achieving negligible loss in retrieval performance.

The first step in an image retrieval pipeline consists of comparing global descriptors from a large database to find a short list of candidate matching images. The more compact the global descriptor, the faster the descriptors can be compared for matching. State-of-the-art global descriptors based on Fisher Vectors are represented with tens of thousands of floating point numbers. While there is significant work on compression of local descriptors, there is relatively little work on compression of high dimensional Fisher Vectors. We study the problem of global descriptor compression in the context of image retrieval, focusing on extremely compact binary representations: 64-1024 bits. Motivated by the remarkable success of deep neural networks in recent literature, we propose a compression scheme based on deeply stacked Restricted Boltzmann Machines (SRBM), which learn lower dimensional non-linear subspaces on which the data lie. We provide a thorough evaluation of several state-of-the-art compression schemes based on PCA, Locality Sensitive Hashing, Product Quantization and greedy bit selection, and show that the proposed compression scheme outperforms all existing schemes.

No abstract

A typical image retrieval pipeline starts with the comparison of global descriptors from a large database to find a short list of candidate matches. A good image descriptor is key to the retrieval pipeline and should reconcile two contradictory requirements: providing recall rates as high as possible and being as compact as possible for fast matching. Following the recent successes of Deep Convolutional Neural Networks (DCNN) for large scale image classification, descriptors extracted from DCNNs are increasingly used in place of the traditional hand crafted descriptors such as Fisher Vectors (FV) with better retrieval performances. Nevertheless, the dimensionality of a typical DCNN descriptor -extracted either from the visual feature pyramid or the fully-connected layers-remains quite high at several thousands of scalar values.In this paper, we propose Unsupervised Triplet Hashing (UTH), a fully unsupervised method to compute extremely compact binary hashes -in the 32-256 bits range-from high-dimensional global descriptors. UTH consists of two successive deep learning steps. First, Stacked Restricted Boltzmann Machines (SRBM), a type of unsupervised deep neural nets, are used to learn binary embedding functions able to bring the descriptor size down to the desired bitrate. SRBMs are typically able to ensure a very high compression rate at the expense of loosing some desirable metric properties of the original DCNN descriptor space. Then, triplet networks, a rank learning scheme based on weight sharing nets is used to fine-tune the binary embedding functions to retain as much as possible of the useful metric properties of the original space. A thorough empirical evaluation conducted on multiple publicly available dataset using DCNN descriptors shows that our method is able to significantly outperform state-of-the-art unsupervised schemes in the target bit range. * Equal contributions from Jie Lin, Olivier Morère and Julie Petta.

Morphology of cell nuclei is a central aspect in many histopathological studies, in particular in breast cancer grading. Therefore, the automatic detection and extraction of cell nuclei from microscopic images obtained from cancer tissue slides is one of the most important problems in digital histopathology.We propose to tackle the problem using a model based on marked point processes (MPP), a methodology for extraction of multiple objects from images. The advantage of MPP based models is their ability to take into account the geometry of objects; and the information about their spatial repartition in the image. Previously, the MPP models have been applied for the extraction of objects of simple geometrical shapes. For histological grading, a morphological criterion known as nuclear pleomorphism corresponding to fine morphological differences between the nuclei is assessed by pathologists. Therefore, the accurate delineation of nuclei became an issue of even greater importance than optimal nuclei detection. Recently, the MPP framework has been defined on the space of arbitrarily-shaped objects allowing more accurate extraction of complex-shaped objects. The nuclei often appear joint or even overlap in histopathological images. The model still allows to extract them as individual joint or overlapping objects without discarding the overlapping parts and therefore without significant loss in delineation precision.We aim to compare the MPP model with two state-of-the-art methods selected from a comprehensive review of the available methods.The experiments are performed using a database of H&E stained breast cancer images covering a wide range of histological grades.