Simulated Stochastic Approximation Annealing for Global Optimization With a Square-Root Cooling Schedule

Liang, Faming; Cheng, Yu‐Shan; Lin, Guang

doi:10.1080/01621459.2013.872993

Cited by 24 publications

(47 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Under the framework of PISAA, we showed that theoretical results of Song et al (2014) for pop-SAMC regarding the asymptotic efficiency of the estimates of the unknown bias weights hold for PISAA as well, and presented theoretical results of Liang et al (2014) for SAA regarding the convergence of the algorithm that hold for PISAA as well. The empirical results confirmed that PISAA produces correct estimates for the unknown bias weights w˚as τ t Ñ τ˚, and that the efficiency of these estimates significantly improves as the population size increases.…”

Section: Discussionmentioning

confidence: 99%

“…tq for tτ t u, which guarantees the global minimum to be reached as the temperature tends to τ˚« 0, lim tÑ8 τ t " τ˚. It is able to locate the minima of each subregion simultaneously (including the global minimum), after a long run, if τ˚is close to 0 (Corollary 3.1 in Liang et al, 2014). It is worth mentioning that the square-root rate is much faster than the logarithmic rate that guarantees convergence in the SA algorithm.…”

Section: Truncation Stepmentioning

confidence: 98%

“…A brief theoretical analysis related to the convergence of PISAA is included in Appendix A, where we show that theoretical results of Song et al 2014 for pop-SAMC hold in the PISAA framework as well, and we present theoretical results in Liang et al (2014) for SAA that hold for PISAA as well. The Theorems A.1, A.2, A.4, and A.5, as well as related conditions on PISAA, are included in the Appendix A.…”

Section: Theoretical Analysis: a Synopsismentioning

confidence: 99%

“…z " p´logp ř m j"1 exppθ pjq n qq; j " 1 : mq for Z " 1. Appropriate conditions under which SAA is a valid adaptive MCMC algorithm that converges to the global minimum are reported in detail in (Conditions A1-A3 in Liang et al, 2014).…”

Section: Truncation Stepmentioning

confidence: 99%

“…In practice, PISAA estimator is expected to outperform the single-chain SAA estimator even when β " 1 because of the so called population effect; the use of multiple-chains to explore the sampling space and approximate the unknown tθ t u. Theorem A.5 implies rigorously that the adjustment process in PISAA is more stable than that in SAA. Liang et al (2014) discussed several practical issues on the implementation of SAA (including the algorithmic settings tπ j u, tγ t u, tτ t u E, tM c u, n ) that are still applicable to PISAA. Here, we adopt these algorithmic settings, i.e.…”

Section: Theoretical Analysis: a Synopsismentioning

confidence: 99%

See 4 more Smart Citations

Parallel and interacting stochastic approximation annealing algorithms for global optimisation

et al. 2016

Self Cite

View full text Add to dashboard Cite

We present the parallel and interacting stochastic approximation annealing (PISAA) algorithm, a stochastic simulation procedure for global optimisation, that extends and improves the stochastic approximation annealing (SAA) by using population Monte Carlo ideas. The standard SAA algorithm guarantees convergence to the global minimum when a square-root cooling schedule is used; however the efficiency of its performance depends crucially on its self-adjusting mechanism. Because its mechanism is based on information obtained from only a single chain, SAA may present slow convergence in complex optimisation problems. The proposed algorithm involves simulating a population of SAA chains that interact each other in a manner that ensures significant improvement of the self-adjusting mechanism and better exploration of the sampling space. Central to the proposed algorithm are the ideas of (i) recycling information from the whole population of Markov chains to design a more accurate/stable self-adjusting mechanism and (ii) incorporating more advanced proposals, such as crossover operations, for the exploration of the sampling space. PISAA presents a significantly improved performance in terms of convergence. PISAA can be implemented in parallel computing environments if available. We demonstrate the good performance of the proposed algorithm on challenging applications including Bayesian network learning and protein folding. Our numerical comparisons suggest that PISAA outperforms the simulated annealing, stochastic approximation annealing, and annealing evolutionary stochastic approximation Monte Carlo especially in high dimensional or rugged scenarios.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Truncation Stepmentioning

confidence: 98%

Section: Theoretical Analysis: a Synopsismentioning

confidence: 99%

Section: Truncation Stepmentioning

confidence: 99%

Section: Theoretical Analysis: a Synopsismentioning

confidence: 99%

See 3 more Smart Citations

Parallel and interacting stochastic approximation annealing algorithms for global optimisation

et al. 2016

Self Cite

View full text Add to dashboard Cite

show abstract

Estimating Convection Parameters in the GFDL CM2.1 Model Using Ensemble Data Assimilation

Zhang

Liu

et al. 2018

J Adv Model Earth Syst

View full text Add to dashboard Cite

Parametric uncertainty in convection parameterization is one major source of model errors that cause model climate drift. Convection parameter tuning has been widely studied in atmospheric models to help mitigate the problem. However, in a fully coupled general circulation model (CGCM), convection parameters which impact the ocean as well as the climate simulation may have different optimal values. This study explores the possibility of estimating convection parameters with an ensemble coupled data assimilation method in a CGCM. Impacts of the convection parameter estimation on climate analysis and forecast are analyzed. In a twin experiment framework, five convection parameters in the GFDL coupled model CM2.1 are estimated individually and simultaneously under both perfect and imperfect model regimes. Results show that the ensemble data assimilation method can help reduce the bias in convection parameters. With estimated convection parameters, the analyses and forecasts for both the atmosphere and the ocean are generally improved. It is also found that information in low latitudes is relatively more important for estimating convection parameters. This study further suggests that when important parameters in appropriate physical parameterizations are identified, incorporating their estimation into traditional ensemble data assimilation procedure could improve the final analysis and climate prediction.

show abstract

TopasOpt: An open‐source library for optimization with Topas Monte Carlo

et al. 2022

View full text Add to dashboard Cite

To describe and test TopasOpt: a free, open-source and extensible library for performing mathematical optimization of Monte Carlo simulations in Topas. Methods: TopasOpt enables any Topas model to be transformed into an optimization problem, and any parameter within the model to be treated as an optimization variable. Three case studies are presented. The starting model consists of a 10 MeV electron beam striking a tungsten target. The resulting bremsstrahlung X-ray spectrum is collimated by a primary and secondary collimator before being scored in a water tank. In the first case study (electron phase space optimization), five parameters describing the electron beam were treated as optimization variables and assigned a random starting value. An objective function was defined based on differences of depth-dose and profiles in water between the original (ground truth) model and a given model generated by TopasOpt. The problem was solved using Bayesian Optimization and the Nelder-Mead method. One hundred iterations were run in each case. In the second case study, (collimator geometry optimization), this process was repeated, but three geometric parameters defining the secondary collimator were treated as optimization variables and assigned random starting values, and forty iterations were run. In the third case study, the optimization was repeated with different number of primary particles to study the effect of noise on convergence. Results: For case 1 (phase space optimization), both optimization algorithms successfully minimized the objective function, with absolute mean differences in profile dose of 0.4% (Bayesian) and 0.3% (Nelder-Mead) and 0.2% in depthdose for both algorithms.The beam energy was recovered to within 1%,however some parameters had relative errors of up to 171% -a result consistent with the known X-ray dose is insensitivity to many electron beam parameters. For case 2 (geometry optimization), absolute mean differences in profile dose were 0.6% (Bayesian) and 0.9% (Nelder-Mead), and 0.5% and 0.9% in depth-dose. The maximum percentage error in any parameter was 9% with Bayesian Optimization and 28% with Nelder-Mead. Finally, the Bayesian Optimization algorithm was demonstrated to be robust to moderate levels of noise; when the standard deviation of the objective function was 16% of the mean, the maximum error in any parameter value was 16%, and the absolute mean difference in dose was 0.9% (profile) and 0.8% (depth-dose).

show abstract

Simulated Stochastic Approximation Annealing for Global Optimization With a Square-Root Cooling Schedule

Cited by 24 publications

References 51 publications

Parallel and interacting stochastic approximation annealing algorithms for global optimisation

Parallel and interacting stochastic approximation annealing algorithms for global optimisation

Estimating Convection Parameters in the GFDL CM2.1 Model Using Ensemble Data Assimilation

TopasOpt: An open‐source library for optimization with Topas Monte Carlo

Contact Info

Product

Resources

About