Machine Learning Assisted Safety Modeling and Analysis of Advanced Reactors

Liu, Yang; Hu, Rui; Dai, Dezhi; Balaprakash, Prasanna; Obabko, Aleksandr

doi:10.2172/1838607

Cited by 2 publications

(1 citation statement)

References 23 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We refer these two search methods as RD+ACBO and GP+ACBO because they both follow a centralized asynchronous architecture. DeepHyper has been used to improve the accuracy of neural networks in several scientific machine learning applications [38], [39], [40], [41]. In addition, we implemented an asynchronous variant of Hyperband (ASHA) [22], [42] based on the pruner API of the Optuna Python package.…”

Section: E Application To Neural Network Hyperparameter Tuningmentioning

confidence: 99%

Asynchronous Distributed Bayesian Optimization at HPC Scale

Égelé¹,

Gouneau²,

Vishwanath³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Bayesian optimization (BO) is a widely used approach for computationally expensive black-box optimization such as simulator calibration and hyperparameter optimization of deep learning methods. In BO, a dynamically updated computationally cheap surrogate model is employed to learn the input-output relationship of the black-box function; this surrogate model is used to explore and exploit the promising regions of the input space. Multipoint BO methods adopt a single manager/multiple workers strategy to achieve high-quality solutions in shorter time. However, the computational overhead in multipoint generation schemes is a major bottleneck in designing BO methods that can scale to thousands of workers. We present an asynchronous-distributed BO (ADBO) method wherein each worker runs a search and asynchronously communicates the input-output values of black-box evaluations from all other workers without the manager. We scale our method up to 4,096 workers and demonstrate improvement in the quality of the solution and faster convergence. We demonstrate the effectiveness of our approach for tuning the hyperparameters of neural networks from the Exascale computing project CANDLE benchmarks.

show abstract