Parallel Bayesian Global Optimization of Expensive Functions

Wang, Jialei; Clark, Scott; Liu, Eric; Frazier, Peter I.

doi:10.1287/opre.2019.1966

Cited by 64 publications

(46 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A common operation in most algorithms is Cholesky decomposition which is used to invert the kernel matrix and is generally O(n 3 ) for n data points, but with care this can be calculated incrementally as new points arrive, reducing the complexity to O(n 2 ) [123]. Several algorithms gain speed-up by implementing part of the algorithm on a GPU, which can be up to 100 times faster than the equivalent single-threaded code [124].…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Bayesian Optimization for Adaptive Experimental Design: A Review

et al. 2020

View full text Add to dashboard Cite

Bayesian optimisation is a statistical method that efficiently models and optimises expensive ''black-box'' functions. This review considers the application of Bayesian optimisation to experimental design, in comparison to existing Design of Experiments (DOE) methods. Solutions are surveyed for a range of core issues in experimental design including: the incorporation of prior knowledge, high dimensional optimisation, constraints, batch evaluation, multiple objectives, multi-fidelity data, and mixed variable types. INDEX TERMS Bayesian methods, design for experiments, design optimization, machine learning algorithms.

show abstract

Section: Discussionmentioning

confidence: 99%

“…• MOE (https://github.com/Yelp/MOE) supports parallel optimisation via multi-point stochastic gradient ascent [124]. Interfaces are provided for Python and C++, and optimisation can be accelerated on GPU hardware.…”

Section: Discussionmentioning

confidence: 99%

Bayesian Optimization for Adaptive Experimental Design: A Review

et al. 2020

View full text Add to dashboard Cite

show abstract

“…SigOpt is an AutoML solution that uses a Bayesian method to construct a feedback mechanism between model output and different values for hyperparameters. Thus, the model can be tuned by selecting the best network parameters to maiximise performance [48].…”

Section: Proposed Methodologymentioning

confidence: 99%

Audio classification using braided convolutional neural networks

Sinha

Awasthi

Ajmera

2020

IET signal process.

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) work surprisingly well and have helped drastically enhance the state-of-the-art techniques in the domain of image classification. The unprecedented success motivated the application of CNNs to the domain of auditory data. Recent publications suggest hidden Markov models and deep neural networks for audio classification. This study aims to achieve audio classification by representing audio as spectrogram images and then use a CNN-based architecture for classification. This study presents an innovative strategy for a CNN-based neural architecture that learns a sparse representation imitating the receptive neurons in the primary auditory cortex in mammals. The feasibility of the proposed CNN-based neural architecture is assessed for audio classification tasks on standard benchmark datasets such as Google Speech Commands datasets (GSCv1 and GSCv2) and the UrbanSound8K dataset (US8K). The proposed CNN architecture, referred to as braided convolutional neural network, achieves 97.15, 95 and 91.9% average recognition accuracy on GSCv1, GSCv2 and US8 K datasets, respectively, outperforming other deep learning architectures.

show abstract

“…In the deep learning field, a number of strategies have been proposed to effectively choose hyper-parameters, including random search (Bergstra and Bengio, 2012), evolutionary strategy such as the NEAT and HyperNEAT algorithms (Stanley and Miikkulainen, 2002;Miikkulainen et al, 2017) and Bayesian optimization (Dewancker et al, 2016b,a;Wang et al, 2016). Nonetheless, manual search is still a commonly used technique, because it is easy to implement and exploits researchers' experience to reduce the number of trials, which is useful when training large networks that require extensive computational resources.…”

Section: Related Workmentioning

confidence: 99%

Optimization of computer aided detection systems: An evolutionary approach

Morra

Coccia

Cerquitelli

2018

Expert Systems with Applications

View full text Add to dashboard Cite

Computer Aided Diagnosis (CAD) systems are designed to aid the radiologist in interpreting medical images. They are usually based on lesion detection and segmentation algorithms whose performance depends on a large number of parameters. While time consuming and sub-optimal, manual adjustment is still widely used to adjust parameter values. Genetic or evolutionary algorithms (GA) are effective optimization methods that mimic biological evolution. Genetic algorithms have been shown to efficiently manage complex search spaces, and can be applied to all kinds of objective functions, including discontinuous, nondifferentiable, or highly nonlinear ones. In this study, we have adopted an evolutionary approach to the problem of parameter optimization. We show that the genetic algorithm is able to effectively converge to a better solution than manual optimization on a case study for digital breast tomosynthesis CAD. Parameter optimization was framed as a constrained optimization problem, where the function to be maximized was defined as weighted sum of sensitivity, false positive rate and segmentation accuracy. A modified Dice coefficient was defined to assess the segmentation quality of individual lesions. Finally, all viable solutions evaluated by the GA were studied by means of exploratory data analysis techniques, such as association rules, to gain useful insight on the strength of the influence of each parameter on overall algorithm performance. We showed that this combination was able to identify multiple ranges of viable solutions with good segmentation accuracy.

show abstract

Parallel Bayesian Global Optimization of Expensive Functions

Abstract: Large-Scale Parallel Bayesian Optimization

Cited by 64 publications

References 36 publications

Bayesian Optimization for Adaptive Experimental Design: A Review

Bayesian Optimization for Adaptive Experimental Design: A Review

Audio classification using braided convolutional neural networks

Optimization of computer aided detection systems: An evolutionary approach

Contact Info

Product

Resources

About