Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support

Cardoso, M. Jorge; Arbel, Tal; Carneiro, Gustavo; Syeda-Mahmood, Tanveer; Tavares, João Manuel R. S.; Moradi, Mehdi; Bradley, Andrew P.; Greenspan, Hayit; Papa, João Paulo; Madabhushi, Anant; Nascimento, Jacinto C.; Cardoso, Jaime S.; Belagiannis, Vasileios; Lu, Zhi; Engenharia, Faculdade de

doi:10.1007/978-3-319-67558-9

Cited by 68 publications

(10 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Accordingly, various methods have been proposed for deriving SCT images, including statistical modeling [3], traditional machine learning [4,5], and multi-atlas based methods [6][7][8][9][10][11][12][13][14]. Recently, increasing interest has been focused on artificial intelligence-based methods, such as deep convolutional neural network (CNN) [15,16,[17][18][19][20][21][22][23][24][25], conditional generative adversarial network (cGAN) [26][27][28][29][30], and cycleconsistent generative adversarial network (cycleGAN) [31][32][33][34][35][36][37].…”

mentioning

confidence: 99%

Magnetic resonance-based synthetic computed tomography images generated using generative adversarial networks for nasopharyngeal carcinoma radiotherapy treatment planning

Peng

Chen

Qin

et al. 2020

Radiotherapy and Oncology

View full text Add to dashboard Cite

Background and purpose: To investigate the feasibility of synthesizing computed tomography (CT) images from magnetic resonance (MR) images using generative adversarial networks (GANs) for nasopharyngeal carcinoma (NPC) intensity-modulated radiotherapy (IMRT) planning. Materials and methods: Conventional T1-weighted MR images and CT images were acquired from 173 NPC patients. The MR and CT images of 28 patients were randomly chosen as the independent tested set. The remaining images were used to build a conditional GAN (cGAN) and a cycle-consistency GAN (cycleGAN). A U-net was used as the generator in cGAN, whereas a residual-Unet was used as the generator in cycleGAN. The cGAN was trained using the deformable registered MR-CT image pairs, whereas the cycleGAN was trained using the unregistered MR and CT images. The generated synthetic CT (SCT) images from cGAN and cycleGAN were compared with the true CT images with respect to their Hounsfield Unit (HU) discrepancy and dosimetric accuracy for NPC IMRT plans. Results: The mean absolute errors within the body were 69.67 ± 9.27 HU and 100.62 ± 7.39 HU for the cGAN and cycleGAN, respectively. The 2%/2-mm c passing rates were (98.68 ± 0.94)% and (98.52 ± 1.13)% for the cGAN and cycleGAN, respectively. Meanwhile, the absolute dose discrepancies within the regions of interest were (0.49 ± 0.24)% and (0.62 ± 0.36)%, respectively. Conclusion: Both cGAN and cycleGAN could swiftly generate accurate SCT volume images from MR images, with high dosimetric accuracy for NPC IMRT planning. cGAN was preferable if high-quality MR-CT image pairs were available.

show abstract

mentioning

confidence: 99%

Magnetic resonance-based synthetic computed tomography images generated using generative adversarial networks for nasopharyngeal carcinoma radiotherapy treatment planning

Peng

Chen

Qin

et al. 2020

Radiotherapy and Oncology

View full text Add to dashboard Cite

show abstract

“…In this work, the negative and positive patches are augmented on-the-fly using horizontal flipping, rotation of up to 30 deg, and rescaling by a factor chosen between 0.75 and 1.25, as commonly used in the literature. 24,25,31,33 We first analyze the performance of the different CNNs for classifying mass and nonmass region in the CBIS-DDSM dataset. The optimizer used is Adam 41 and the batch size is 128 (for a GPU of 12 GB).…”

Section: Cnn Trainingmentioning

confidence: 99%

“…In another work, breast abnormalities (masses, microcalcifications) were simultaneously detected using a faster R-CNN model and a CNN-based classifier 25 obtaining a TPR of 0.93 at 0.56 FPI for mass mammograms using a subset of the INbreast database. Recently, Ribli et al 26 used fast R-CNN for the classification and detection of malignant and benign lesions with a TPR of 0.90 at 0.3 FPI, using a subset of the INbreast database with lesions.…”

Section: Introductionmentioning

confidence: 99%

Automatic mass detection in mammograms using deep convolutional neural networks

et al. 2019

View full text Add to dashboard Cite

With recent advances in the field of deep learning, the use of convolutional neural networks (CNNs) in medical imaging has become very encouraging. The aim of our paper is to propose a patch-based CNN method for automated mass detection in full-field digital mammograms (FFDM). In addition to evaluating CNNs pretrained with the ImageNet dataset, we investigate the use of transfer learning for a particular domain adaptation. First, the CNN is trained using a large public database of digitized mammograms (CBIS-DDSM dataset), and then the model is transferred and tested onto the smaller database of digital mammograms (INbreast dataset). We evaluate three widely used CNNs (VGG16, ResNet50, InceptionV3) and show that the InceptionV3 obtains the best performance for classifying the mass and nonmass breast region for CBIS-DDSM. We further show the benefit of domain adaptation between the CBIS-DDSM (digitized) and INbreast (digital) datasets using the InceptionV3 CNN. Mass detection evaluation follows a fivefold cross-validation strategy using free-response operating characteristic curves. Results show that the transfer learning from CBIS-DDSM obtains a substantially higher performance with the best true positive rate (TPR) of 0.98 AE 0.02 at 1.67 false positives per image (FPI), compared with transfer learning from ImageNet with TPR of 0.91 AE 0.07 at 2.1 FPI. In addition, the proposed framework improves upon mass detection results described in the literature on the INbreast database, in terms of both TPR and FPI.

show abstract

“…17 Although primarily developed to segment neuronal structures in electron microscopy stacks, it has since been successfully applied to a variety of biomedical image segmentation tasks. 48,83 Although these early studies show great promise, the most common training and testing sets from challenges are still limited in scope compared with the range of diseases that would be seen in clinical practice. Thus, these algorithms must yet prove themselves in the real world, as larger and more routine clinical data sets become available.…”

Section: Developing Communities Competitions and The Challenges Of mentioning

confidence: 99%

Machine Learning and Deep Neural Networks in Thoracic and Cardiovascular Imaging

Retson

Besser

Sall³

et al. 2019

Journal of Thoracic Imaging

View full text Add to dashboard Cite

Advances in technology have always had the potential and opportunity to shape the practice of medicine, and in no medical specialty has technology been more rapidly embraced and adopted than radiology. Machine learning and deep neural networks promise to transform the practice of medicine, and, in particular, the practice of diagnostic radiology. These technologies are evolving at a rapid pace due to innovations in computational hardware and novel neural network architectures. Several cutting-edge postprocessing analysis applications are actively being developed in the fields of thoracic and cardiovascular imaging, including applications for lesion detection and characterization, lung parenchymal characterization, coronary artery assessment, cardiac volumetry and function, and anatomic localization. Cardiothoracic and cardiovascular imaging lies at the technological forefront of radiology due to a confluence of technical advances. Enhanced equipment has enabled computed tomography and magnetic resonance imaging scanners that can safely capture images that freeze the motion of the heart to exquisitely delineate fine anatomic structures. Computing hardware developments have enabled an explosion in computational capabilities and in data storage. Progress in software and fluid mechanical models is enabling complex 3D and 4D reconstructions to not only visualize and assess the dynamic motion of the heart, but also quantify its blood flow and hemodynamics. And now, innovations in machine learning, particularly in the form of deep neural networks, are enabling us to leverage the increasingly massive data repositories that are prevalent in the field. Here, we discuss developments in machine learning techniques and deep neural networks to highlight their likely role in future radiologic practice, both in and outside of image interpretation and analysis. We discuss the concepts of validation, generalizability, and clinical utility, as they pertain to this and other new technologies, and we reflect upon the opportunities and challenges of bringing these into daily use.

show abstract

Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support

Cited by 68 publications

References 6 publications

Magnetic resonance-based synthetic computed tomography images generated using generative adversarial networks for nasopharyngeal carcinoma radiotherapy treatment planning

Magnetic resonance-based synthetic computed tomography images generated using generative adversarial networks for nasopharyngeal carcinoma radiotherapy treatment planning

Automatic mass detection in mammograms using deep convolutional neural networks

Machine Learning and Deep Neural Networks in Thoracic and Cardiovascular Imaging

Contact Info

Product

Resources

About