Efficient Deep Learning Hyperparameter Tuning Using Cloud Infrastructure: Intelligent Distributed Hyperparameter Tuning with Bayesian Optimization in the Cloud

Ranjit, Mercy; Ganapathy, Gopinath; Sridhar, Kalaivani; Arumugham, Vikram

doi:10.1109/cloud.2019.00097

Cited by 35 publications

(15 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hyperparamater Tuning provides an optimal parameter search method for the proposed Convolutional Neural Network model. CNN combined with Hyperparameter Tuning requires to set the parameters, such as: kernel size, steps, number of channels, and number of dropouts [24]. Hyperparameter Tuning provides the best combination of parameters to the model demonstrating maximum results [25].…”

Section: Hyperparameter Tuningmentioning

confidence: 99%

Convolutional Neural Network with Hyperparameter Tuning for Brain Tumor Classification

Minarno

Mandiri

Munarko

et al. 2021

KINETIK

View full text Add to dashboard Cite

Brain tumor has been acknowledged as the most dangerous disease through all its circles. Early identification of tumor disease is considered pivotal to identify the spread of brain tumors in administering the appropriate treatment. This study proposes a Convolutional Neural Network method to detect brain tumor on MRI images. The 3264 datasets were undertaken in this study with detailed images of Glioma tumor (926 images), Meningioma tumors (937 images), pituitary tumors (901 images), and other with no-tumors (500 images). The application of CNN method combined with Hyperparameter Tuning is proposed to achieve optimal results in classifying the brain tumor types. Hyperparameter Tuning acts as a navigator to achieve the best parameters in the proposed CNN model. In this study, the model testing was conducted with three different scenarios. The result of brain tumor classification depicts an accuracy of 96% in the third model testing scenario.

show abstract

Section: Hyperparameter Tuningmentioning

confidence: 99%

Convolutional Neural Network with Hyperparameter Tuning for Brain Tumor Classification

Minarno

Mandiri

Munarko

et al. 2021

KINETIK

View full text Add to dashboard Cite

show abstract

“…3) Distributed Hyperparameter tuning: Since the begining when the use of neural networks was first introduced, the hyperparameter optimization or tuning has been essential to improve their performance [20]. It is also well known, that it is a tedious and slow process, for that reason several studies on distributing it and thus increase its performance have been carried out [21], [22]. Furthermore, it has also been proposed on medical image diagnosis [23], [24], [25], but in these studies their focus is not on efficiency.…”

Section: State Of the Artmentioning

confidence: 99%

Distributing Deep Learning Hyperparameter Tuning for 3D Medical Image Segmentation

Berral¹,

Oriol²,

Domínguez³

et al. 2021

Preprint

View full text Add to dashboard Cite

Most research on novel techniques for 3D Medical Image Segmentation (MIS) is currently done using Deep Learning with GPU accelerators. The principal challenge of such technique is that a single input can easily cope computing resources, and require prohibitive amounts of time to be processed. Distribution of deep learning and scalability over computing devices is an actual need for progressing on such research field. Conventional distribution of neural networks consist in "data parallelism", where data is scattered over resources (e.g., GPUs) to parallelize the training of the model. However, "experiment parallelism" is also an option, where different training processes (i.e., on a hyper-parameter search) are parallelized across resources. While the first option is much more common on 3D image segmentation, the second provides a pipeline design with less dependence among parallelized processes, allowing overhead reduction and more potential scalability. In this work we present a design for distributed deep learning training pipelines, focusing on multinode and multi-GPU environments, where the two different distribution approaches are deployed and benchmarked. We take as proof of concept the 3D U-Net architecture, using the MSD Brain Tumor Segmentation dataset, a state-of-art problem in medical image segmentation with high computing and space requirements. Using the BSC MareNostrum supercomputer as benchmarking environment, we use TensorFlow and Ray as neural network training and experiment distribution platforms. We evaluate the experiment speed-up when parallelizing, showing the potential for scaling out on GPUs and nodes. Also comparing the different parallelism techniques, showing how experiment distribution leverages better such resources through scaling, e.g. by a speed-up factor from x12 to x14 using 32 GPUs. Finally, we provide the implementation of the design open to the community, and the non-trivial steps and methodology for adapting and deploying a MIS case as the here presented.

show abstract

“…The purpose of hyperparameter tuning is to get optimal hyperparameters [33]. The tuning hyperparameters in this study include the number of hidden units, dropouts, and learning rates.…”

Section: B Hyperparameter Tuning Random Searchmentioning

confidence: 99%

Customer Decision Prediction Using Deep Neural Network on Telco Customer Churn Data

Nalatissifa

Pardede

2021

J. Elektron. dan Telekomun.

View full text Add to dashboard Cite

Customer churn is the most important problem in the business world, especially in the telecommunications industry, because it greatly influences company profits. Getting new customers for a company is much more difficult and expensive than retaining existing customers. Machine learning, part of data mining, is a sub-field of artificial intelligence widely used to make predictions, including predicting customer churn. Deep neural network (DNN) has been used for churn prediction, but selecting hyperparameters in modeling requires more time and effort, making the process more challenging for the researcher. Therefore, the purpose of this study is to propose a better architecture for the DNN algorithm by using a hard tuner to obtain more optimal hyperparameters. The tuning hyperparameter used is random search in determining the number of nodes in each hidden layer, dropout, and learning rate. In addition, this study also uses three variations of the number of hidden layers, two variations of the activation function, namely rectified linear unit (ReLu) and Sigmoid, then uses five variations of the optimizer (stochastic gradient descent (SGD), adaptive moment estimation (Adam), adaptive gradient algorithm (Adagrad), Adadelta, and root mean square propagation (RMSprop)). Experiments show that the DNN algorithm using hyperparameter tuning random search produces a performance value of 83.09 % accuracy using three hidden layers, the number of nodes in each hidden layer is [20, 35, 15], using the RMSprop optimizer, dropout 0.1, the learning rate is 0.01, with the fastest tuning time of 21 seconds. Better than modeling using k-nearest neighbor (K-NN), random forest (RF), and decision tree (DT) as comparison algorithms.

show abstract

Efficient Deep Learning Hyperparameter Tuning Using Cloud Infrastructure: Intelligent Distributed Hyperparameter Tuning with Bayesian Optimization in the Cloud

Cited by 35 publications

References 1 publication

Convolutional Neural Network with Hyperparameter Tuning for Brain Tumor Classification

Convolutional Neural Network with Hyperparameter Tuning for Brain Tumor Classification

Distributing Deep Learning Hyperparameter Tuning for 3D Medical Image Segmentation

Customer Decision Prediction Using Deep Neural Network on Telco Customer Churn Data

Contact Info

Product

Resources

About