Deep-learning based, automated segmentation of macular edema in optical coherence tomography

Lee, Cecilia S.; Tyring, Ariel J.; Deruyter, Nicolaas P.; Wu, Yue; Rokem, Ariel; Lee, Aaron

doi:10.1364/boe.8.003440

Cited by 306 publications

(187 citation statements)

References 29 publications

Supporting

Mentioning

165

Contrasting

Unclassified

Order By: Relevance

“…For example even an expert might not provide the exact same ratings each time they see the same image, while an algorithm will. This is in line with findings from (Lee et al, 2017c), showing that the agreement between an algorithm and any one expert can be equivalent to agreement between any pair of experts. We have demonstrated that while an individual citizen scientists may not provide reliable results, by intelligently combining a crowd with machine learning, and keeping an expert in the loop to monitor results, decisions can be accurately scaled to meet the demands of Big Data.…”

Section: Scaling Expertise Through Interactions Between Experts Citisupporting

confidence: 90%

“…Modeled loosely on the human visual system, CNNs can be trained for a variety of image classification and segmentation tasks using the same architecture. For example, the U-Net (Ronneberger et al, 2015) which was originally built for segmentation of neurons in electron microscope images, has also been adapted to segment macular edema in optical coherence tomography images (Lee et al, 2017b), to segment breast and fibroglandular tissue (Dalmış et al, 2017), and a 3D adaptation was developed to segment the Xenopus kidney (Ç içek et al, 2016). Transfer learning is another broadly applicable deep learning technique, where a number of layers from pretrained network are retrained for a different use case.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Combining citizen science and deep learning to amplify expertise in neuroimaging

Keshavan

Yeatman

Rokem

2018

Preprint

Self Cite

View full text Add to dashboard Cite

Research in many fields has become increasingly reliant on large and complex datasets. "Big Data" holds untold promise to rapidly advance science by tackling new questions that cannot be answered with smaller datasets. While powerful, research with Big Data poses unique challenges, as many standard lab protocols rely on experts examining each one of the samples. This is not feasible for large-scale datasets because manual approaches are time-consuming and hence difficult to scale. Meanwhile, automated approaches lack the accuracy of examination by highly trained scientists and this may introduce major errors, sources of noise, and unforeseen biases into these large and complex datasets. Our proposed solution is to 1) start with a small, expertly labelled dataset, 2) amplify labels through web-based tools that engage citizen scientists, and 3) train machine learning on amplified labels to emulate expert decision making. As a proof of concept, we developed a system to quality control a large dataset of three-dimensional magnetic resonance images (MRI) of human brains. An initial dataset of 200 brain images labeled by experts were amplified by citizen scientists to label 722 brains, with over 80,000 ratings done through a simple web interface. A deep learning algorithm was then trained to predict data quality, based on a combination of the citizen scientist labels that accounts for differences in the quality of classification by different citizen scientists. In an ROC analysis (on left out test data), the deep learning network performed as well as a state-of-the-art, specialized algorithm (MRIQC) for quality control of T1-weighted images, each with an area under the curve of 0.99. Finally, as a specific practical application of the method, we explore how brain image quality relates to the replicability of a well established relationship between brain volume and age over development. Combining citizen science and deep learning can generalize and scale expert decision making; this is particularly important in emerging disciplines where specialized, automated tools do not already exist.

show abstract

Section: Scaling Expertise Through Interactions Between Experts Citisupporting

confidence: 90%

Section: Introductionmentioning

confidence: 99%

Combining citizen science and deep learning to amplify expertise in neuroimaging

Keshavan

Yeatman

Rokem

2018

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…A convolutional neural network was trained to learn the mapping between different MRI images in a training set (n=338). We used a U-net architecture (7) , with loss evaluated on "perceptual loss" (9) : the activation of the first layer of a pretrained VGG16 network (11) , a cost function that induces image similarity and prevents over-smoothing. Training used the Adam optimizer (learning rate: 0.0002).…”

Section: Methodsmentioning

confidence: 99%

“…We used deep learning (DL) algorithm to learn the complex mappings between different MRI contrasts. DL has been successful in many different domains (3) , and there are several previous applications of DL in biomedical imaging (4) , including classification of disease states (5) and segmentation of tissue-type (6) lesions (7) and tumors (8) . DL algorithms that take an image as input and generate another image as output have been used for style-transfer: retaining the content of an image, while altering its style to match the style of another image (9) .…”

Section: Introductionmentioning

confidence: 99%

MRI2MRI: A deep convolutional network that accurately transforms between brain MRI contrasts

Xiao

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

Different brain MRI contrasts represent different tissue properties and are sensitive to different artifacts. The relationship between different contrasts is therefore complex and nonlinear. We developed a deep convolutional network that learns the mapping between different MRI contrasts. Using a publicly available dataset, we demonstrate that this algorithm accurately transforms between T1-and T2-weighted images, proton density images, time-of-flight angiograms, and diffusion MRI images. We demonstrate that these transformed images can be used to improve spatial registration between MR images of different contrasts. INTRODUCTION:MRI creates images that are sensitive to different aspects of the tissue, and susceptible to different imaging artifacts. The relationships between different imaging contrasts are nonlinear and spatially-and tissue-dependent (1) . This poses several difficulties in the interpretation of multi-modal MRI. For example, analysis that requires accurate registration of images into the same coordinate frame currently requires the use of algorithms that can match images with different contrasts (2) . While this often works well, these algorithms can be over-sensitive to large, prominent features, such as edges of the tissue, and are more error-prone when images have low SNR, or represent very different features. Moreover, a better understanding of complementary information provided in different contrasts will allow better characterization of tissue properties.

show abstract

“…In der hier vorgestellten Studie [3] wird ein solcher Deep-Learning-Algorithmus für die automatische Erkennung von typischen Veränderungen bei OCT-Bildgebungsdaten verwendet, die für das Vorliegen einer altersbedingten Makuladegeneration (AMD) sprechen. Dazu verglichen die Arbeitsgruppe Lee et al 52 690 retinale OCT-Scans von Patienten mit Normalbefunden mit 48 312 OCT-Scans von Patienten, bei denen eine AMD vorlag.…”

Section: Transfer In Die Praxis Von Dr Sebastian Siebemann (Köln)unclassified

Deep-Learning-gestützte Computer-Algorithmen zur Diagnosestellung in der Augenheilkunde

Siebelmann¹

2018

Kompass Ophthalmol

View full text Add to dashboard Cite

Purpose: The advent of electronic medical records (EMRs) with large electronic imaging databases along with advances in deep neural networks with machine learning has provided a unique opportunity to achieve milestones in automated image analysis. Optical coherence tomography is the most common imaging modality in ophthalmology and represents a dense and rich data set when combined with labels derived from the EMR. We sought to determine whether deep learning could be utilized to distinguish normal OCT images from images from patients with age-related macular degeneration (AMD). Design: EMR and OCT database study. Subjects: Normal and AMD patients who underwent macular OCT. Methods: Automated extraction of an OCT database was performed and linked to clinical end points from the EMR. Optical coherence tomography scans of the macula were obtained by Heidelberg Spectralis, and each OCT scan was linked to EMR clinical end points extracted from EPIC. The central 11 images were selected from each OCT scan of 2 cohorts of patients: normal and AMD. Cross-validation was performed using a random subset of patients. Receiver operating characteristic (ROC) curves were constructed at an independent image level, macular OCT level, and patient level. Main Outcome Measure: Area under the ROC curve. Results: Of a recent extraction of 2.6 million OCT images linked to clinical data points from the EMR, 52 690 normal macular OCT images and 48 312 AMD macular OCT images were selected. A deep neural network was trained to categorize images as either normal or AMD. At the image level, we achieved an area under the ROC curve of 92.78% with an accuracy of 87.63%. At the macula level, we achieved an area under the ROC curve of 93.83% with an accuracy of 88.98%. At a patient level, we achieved an area under the ROC curve of 97.45% with an accuracy of 93.45%. Peak sensitivity and specificity with optimal cutoffs were 92.64% and 93.69%, respectively. Conclusions: The deep learning technique achieves high accuracy and is effective as a new image classification technique. These findings have important implications in utilizing OCT in automated screening and the development of computer-aided diagnosis tools in the future.

show abstract

Deep-learning based, automated segmentation of macular edema in optical coherence tomography

Cited by 306 publications

References 29 publications

Combining citizen science and deep learning to amplify expertise in neuroimaging

Combining citizen science and deep learning to amplify expertise in neuroimaging

MRI2MRI: A deep convolutional network that accurately transforms between brain MRI contrasts

Deep-Learning-gestützte Computer-Algorithmen zur Diagnosestellung in der Augenheilkunde

Contact Info

Product

Resources

About