Deep learning methods for digital pathology analysis are an effective way to address multiple clinical questions, from diagnosis to prediction of treatment outcomes. These methods have also been used to predict gene mutations from pathology images, but no comprehensive evaluation of their potential for extracting molecular features from histology slides has yet been performed. We show that HE2RNA, a model based on the integration of multiple data modes, can be trained to systematically predict RNA-Seq profiles from whole-slide images alone, without expert annotation. Through its interpretable design, HE2RNA provides virtual spatialization of gene expression, as validated by CD3-and CD20-staining on an independent dataset. The transcriptomic representation learned by HE2RNA can also be transferred on other datasets, even of small size, to increase prediction performance for specific molecular phenotypes. We illustrate the use of this approach in clinical diagnosis purposes such as the identification of tumors with microsatellite instability.

BaCKgRoUND aND aIMS: Standardized and robust risk-stratification systems for patients with hepatocellular carcinoma (HCC) are required to improve therapeutic strategies and investigate the benefits of adjuvant systemic therapies after curative resection/ablation. appRoaCH aND ReSUltS: In this study, we used two deep-learning algorithms based on whole-slide digitized histological slides (whole-slide imaging; WSI) to build models for predicting survival of patients with HCC treated by surgical resection. Two independent series were investigated: a discovery set (Henri Mondor Hospital, n = 194) used to develop our algorithms and an independent validation set (The Cancer Genome Atlas [TCGA], n = 328). WSIs were first divided into small squares ("tiles"), and features were extracted with a pretrained convolutional neural network (preprocessing step). The first deep-learning-based algorithm ("SCHMOWDER") uses an attention mechanism on tumoral areas annotated by a pathologist whereas the second ("CHOWDER") does not require human expertise. In the discovery set, c-indices for survival prediction of SCHMOWDER and CHOWDER reached 0.78 and 0.75, respectively. Both models outperformed a composite score incorporating all baseline variables associated with survival. Prognostic value of the models was further validated in the TCGA data set, and, as observed in the discovery series, both models had a higher discriminatory power than a score combining all baseline variables associated with survival. Pathological review showed that the tumoral areas most predictive of poor survival were characterized by vascular spaces, the macrotrabecular architectural pattern, and a lack of immune infiltration. CoNClUSIoNS: This study shows that artificial intelligence can help refine the prediction of HCC prognosis. It highlights the importance of pathologist/machine interactions for the construction of deep-learning algorithms that benefit from expert knowledge and allow a biological understanding of their output.

No abstract

No abstract

No abstract

In this work we design a neural network for recognizing emotions in speech, using the IEMOCAP dataset. Following the latest advances in audio analysis, we use an architecture involving both convolutional layers, for extracting high-level features from raw spectrograms, and recurrent ones for aggregating long-term dependencies. We examine the techniques of data augmentation with vocal track length perturbation, layer-wise optimizer adjustment, batch normalization of recurrent layers and obtain highly competitive results of 64.5% for weighted accuracy and 61.7% for unweighted accuracy on four emotions.

PurposeDiabetic retinopathy (DR) is the major cause of blindness in the working‐age population. With an increasing number of diabetic patients worldwide, automated screening tools become indispensable. Recent progress in machine learning and image analysis enables efficient automated screening.MethodsDreamUp Vision uses state‐of the art technology based on deep‐learning. Our algorithm was trained on over 70,000 labeled retinal images. Images were graded by ophthalmologists as follows: 0 (no retinopathy), 1 (mild non proliferative DR), 2 (moderate non proliferative DR), 3 (severe non proliferative DR) and 4 (proliferative retinopathy). Each patient in the dataset is represented by two images of left and right eyes. Grading is done for each eye image separately. Our algorithm performs quick and reliable detection of anomalies in retinal images, diagnoses their stage of diabetic retinopathy and provides the location of the anomalies detected in the pictures. We consider a patient as referable if the DR stage is between 2 and 4, otherwise we consider the patient as non‐referable. We evaluate our model on over 10,000 fundus images from 5,000 patients taken from the Kaggle DR Detection Challenge dataset, provided by California Healthcare Foundation.ResultsOur algorithm achieves an area under the receiver operating characteristic curve AUROC of 0.946 with 96.2% sensitivity (95% CI: 95.8–96.5) and 66.6% specificity (95% CI: 65.7–67.5) for identifying referable DR on the Kaggle dataset.ConclusionsThe performances we have obtained enable a reliable automated DR screening. As the amount of available labeled data grows and given our technology's ability to learn from labeled images, we believe that significant performance improvement can be achieved. The same process can be applied to the detection of other eye diseases as well.

^,⋕ : These authors contributed equally.Deep learning methods for digital pathology analysis have proved an effective way to address multiple clinical questions, from diagnosis to prognosis and the prediction of treatment outcomes. They have also recently been used to predict gene mutations from pathology images, but no comprehensive evaluation of their potential for extracting molecular features from histology slides, has yet been performed. We propose a novel approach based on the integration of multiple data modes, and show that our deep learning model HE2RNA can be trained to predict systematically RNA-Seq profiles from whole-slide images alone, without the need for expert annotation. The model facilitates the virtual spatialization of gene expression, as validated by double-staining in an independent dataset. The results can therefore be interpreted in detail and this model opens up new opportunities for virtual staining. Finally, the transcriptomic representation learned by the model could be could be used to improve performances for other clinical tasks, particularly for small datasets. For example we studied the problem of predicting microsatellite instability from Hematoxylin & Eosin (H&E)-stained images. Greater prediction ability was achieved in such a realistic framework.