Past, present, and future trend of GPU computing in deep learning on medical images

Haryanto, Toto; Suhartanto, Heru; Lie, X.

doi:10.1109/icacsis.2017.8355007

“…This paper focuses on identifying reduced-order transfer function models for a gasier with a minimum IAE and ISE error criterion using a GA. The lower order transfer functions obtained using the Genetic Algorithm are found to be superior to those obtained using the RGA loop pairing and the algebraic method proposed, respectively, by Haryanto and Sivakumar et al 63,64…”

Section: Identication Of Biomass Gasication Systemmentioning

confidence: 85%

Recent advances in dynamic modeling and control studies of biomass gasification for production of hydrogen rich syngas

Hussain¹,

Ali²,

Raza

³

et al. 2023

RSC Adv.

6

0

View full text Add to dashboard Cite

show abstract

“…In data parallelism, a batch of data is split across the devices and each one computes a mini-batch; it's the most used and is demonstrated as the most efficient and preferred approach whereas either the model or a sample of data can be fed into memory. All these approaches try to solve the common problem of memory limitations when using heavy datasets or models, hence, specially in medical images their application has been also studied [17]. Spatial parallelism has been applied for high resolution medical image analysis [18].…”

Section: State Of the Artmentioning

confidence: 99%

Distributing Deep Learning Hyperparameter Tuning for 3D Medical Image Segmentation

Berral

¹

,

Oriol

²

,

Domínguez

³

et al. 2022

2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)

3

0

View full text Add to dashboard Cite

Most research on novel techniques for 3D Medical Image Segmentation (MIS) is currently done using Deep Learning with GPU accelerators. The principal challenge of such technique is that a single input can easily cope computing resources, and require prohibitive amounts of time to be processed. Distribution of deep learning and scalability over computing devices is an actual need for progressing on such research field. Conventional distribution of neural networks consist in "data parallelism", where data is scattered over resources (e.g., GPUs) to parallelize the training of the model. However, "experiment parallelism" is also an option, where different training processes (i.e., on a hyper-parameter search) are parallelized across resources. While the first option is much more common on 3D image segmentation, the second provides a pipeline design with less dependence among parallelized processes, allowing overhead reduction and more potential scalability. In this work we present a design for distributed deep learning training pipelines, focusing on multinode and multi-GPU environments, where the two different distribution approaches are deployed and benchmarked. We take as proof of concept the 3D U-Net architecture, using the MSD Brain Tumor Segmentation dataset, a state-of-art problem in medical image segmentation with high computing and space requirements. Using the BSC MareNostrum supercomputer as benchmarking environment, we use TensorFlow and Ray as neural network training and experiment distribution platforms. We evaluate the experiment speed-up when parallelizing, showing the potential for scaling out on GPUs and nodes. Also comparing the different parallelism techniques, showing how experiment distribution leverages better such resources through scaling, e.g. by a speed-up factor from x12 to x14 using 32 GPUs. Finally, we provide the implementation of the design open to the community, and the non-trivial steps and methodology for adapting and deploying a MIS case as the here presented.

show abstract

“…In data parallelism, a batch of data is split across the devices and each one computes a mini-batch; it's the most used and is demonstrated as the most efficient and preferred approach whereas either the model or a sample of data can be fed into memory. All these approaches try to solve the common problem of memory limitations when using heavy datasets or models, hence, specially in medical images their application has been also studied [11]. Spatial parallelism has been applied for high resolution medical image analysis [12].…”

Section: State Of the Artmentioning

confidence: 99%

Distributing Deep Learning Hyperparameter Tuning for 3D Medical Image Segmentation

Berral¹,

Oriol²,

Domínguez³

et al. 2021

Preprint

View full text Add to dashboard Cite

Most research on novel techniques for 3D Medical Image Segmentation (MIS) is currently done using Deep Learning with GPU accelerators. The principal challenge of such technique is that a single input can easily cope computing resources, and require prohibitive amounts of time to be processed. Distribution of deep learning and scalability over computing devices is an actual need for progressing on such research field. Conventional distribution of neural networks consist in "data parallelism", where data is scattered over resources (e.g., GPUs) to parallelize the training of the model. However, "experiment parallelism" is also an option, where different training processes (i.e., on a hyper-parameter search) are parallelized across resources. While the first option is much more common on 3D image segmentation, the second provides a pipeline design with less dependence among parallelized processes, allowing overhead reduction and more potential scalability. In this work we present a design for distributed deep learning training pipelines, focusing on multinode and multi-GPU environments, where the two different distribution approaches are deployed and benchmarked. We take as proof of concept the 3D U-Net architecture, using the MSD Brain Tumor Segmentation dataset, a state-of-art problem in medical image segmentation with high computing and space requirements. Using the BSC MareNostrum supercomputer as benchmarking environment, we use TensorFlow and Ray as neural network training and experiment distribution platforms. We evaluate the experiment speed-up when parallelizing, showing the potential for scaling out on GPUs and nodes. Also comparing the different parallelism techniques, showing how experiment distribution leverages better such resources through scaling, e.g. by a speed-up factor from x12 to x14 using 32 GPUs. Finally, we provide the implementation of the design open to the community, and the non-trivial steps and methodology for adapting and deploying a MIS case as the here presented.

show abstract

Past, present, and future trend of GPU computing in deep learning on medical images

Cited by 6 publications

References 22 publications

Recent advances in dynamic modeling and control studies of biomass gasification for production of hydrogen rich syngas

Recent advances in dynamic modeling and control studies of biomass gasification for production of hydrogen rich syngas

Distributing Deep Learning Hyperparameter Tuning for 3D Medical Image Segmentation

Distributing Deep Learning Hyperparameter Tuning for 3D Medical Image Segmentation

Contact Info

Product

Resources

About