Adaptive stochastic approximation by the simultaneous perturbation method

Spall, James C.

doi:10.1109/tac.2000.880982

Cited by 326 publications

(167 citation statements)

References 44 publications

Supporting

Mentioning

156

Contrasting

Order By: Relevance

“…Like the standard SPSA algorithm, the ASP algorithm requires only a small number of loss function (or gradient, if relevant) measurements per iteration -independent of the problem dimension -to adaptively estimate the Hessian and parameters of primary interest. Further information is available at Spall (2000) or Spall (2003, Sect. 7.8).…”

Section: Copyright Springer Heidelberg 2004mentioning

confidence: 99%

Stochastic Optimization

Spall

2011

Handbook of Computational Statistics

View full text Add to dashboard Cite

Section: Copyright Springer Heidelberg 2004mentioning

confidence: 99%

Stochastic Optimization

Spall

2011

Handbook of Computational Statistics

View full text Add to dashboard Cite

“…In our implementation, we have been using the Simultaneous Perturbation Stochastic Approximation (SPSA) introduced by Spall [30]. The main feature of the SPSA algorithm is to provide a stochastic and fast estimate of the cost function gradient in a finite difference way by perturbing simultaneously the entire set of parameters.…”

Section: Group-wise Affine Alignmentmentioning

confidence: 99%

Unbiased group-wise alignment by iterative central tendency estimations

Craene

Macq

Marqués

et al. 2008

Math. Model. Nat. Phenom.

View full text Add to dashboard Cite

Abstract. This paper introduces a new approach for the joint alignment of a large collection of segmented images into the same system of coordinates while estimating at the same time an optimal common coordinate system. The atlas resulting from our group-wise alignment algorithm is obtained as the hidden variable of an Expectation-Maximization (EM) estimation. This is achieved by identifying the most consistent label across the collection of images at each voxel in the common frame of coordinates.In an iterative process, each subject is iteratively aligned with the current probabilistic atlas until convergence of the estimated atlas is reached. Two different transformation models are successively applied in the alignment process: an affine transformation model and a dense non-rigid deformation field. The metric for both transformation models is the mutual information that is computed between the probabilistic atlas and each subject. This metric is optimized in the affine alignment step using a gradient based stochastic optimization (SPSA) and with a variational approach to estimate the non-rigid atlas to subject transformations.A first advantage of our method is that the computational cost increases linearly with the number of subjects in the database. This method is therefore particularly suited for a large number of subjects. Another advantage is that, when computing the common coordinate system, the estimation algorithm identifies weights for each subject on the basis of the typicality of the segmentation. This makes the common coordinate system robust to outliers in the population.Several experiments are presented in this paper to validate our atlas construction method on a population of 80 brain images segmented into 4 labels (background, white and gray matters Unbiased group-wise alignment and ventricles). First, the 80 subjects were aligned using affine and dense non-rigid deformation models. The results are visually assessed by examining how the population converges closer to a central tendency when the deformation model allows more degrees of freedom (from affine to dense non-rigid field). Second, the stability of the atlas construction procedure for various sizes of population was investigated by starting from a subset of the total population which was incrementally augmented until the total population of 80 subjects was reached. Third, the consistency of our group-wise reference (hidden variable of the EM algorithm) was also compared to the choice of an arbitrary subject for a subset of 10 subjects. According to William's index, our reference choice performed favorably. Finally, the performance of our algorithm was quantified on a synthetic population of 10 subjects (generated using random B-Spline transformations) using a global overlap measure for each label. We also measured the robustness of this measure to the introduction of noisy subjects in the population.

show abstract

“…Classically, second and higher-order methods are suggested as the natural way to approach this issue. Gradient-free counterparts of such methods (e.g., 2SPSA) have been investigated both theoretically and empirically by Spall (2000) and later by Dippon (2003). The excessive computational complexity and memory requirements of these methods, however, make them less attractive for practitioners.…”

Section: Multiple Evaluations Vs Multiple Perturbationsmentioning

confidence: 99%

Universal parameter optimisation in games based on SPSA

Kocsis

Szepesvári

2006

Mach Learn

View full text Add to dashboard Cite

Most game programs have a large number of parameters that are crucial for their performance. While tuning these parameters by hand is rather difficult, efficient and easy to use generic automatic parameter optimisation algorithms are known only for special problems such as the adjustment of the parameters of an evaluation function. The SPSA algorithm (Simultaneous Perturbation Stochastic Approximation) is a generic stochastic gradient method for optimising an objective function when an analytic expression of the gradient is not available, a frequent case in game programs. Further, SPSA in its canonical form is very easy to implement. As such, it is an attractive choice for parameter optimisation in game programs, both due to its generality and simplicity. The goal of this paper is twofold: (i) to introduce SPSA for the game programming community by putting it into a game-programming perspective, and (ii) to propose and discuss several methods that can be used to enhance the performance of SPSA. These methods include using common random numbers and antithetic variables, a combination of SPSA with RPROP, and the reuse of samples of previous performance evaluations. SPSA with the proposed enhancements was tested in some large-scale experiments on tuning the parameters of an opponent model, a policy and an evaluation function in our poker program, MCRAISE. Whilst SPSA with no enhancements failed to make progress using the allocated resources, SPSA with the enhancements proved to be competitive with other methods, including TD-learning; increasing the average payoff per game by as large as 0.19 times the size of the amount of the small bet. From the experimental study, we conclude that the use of an appropriately enhanced variant of SPSA for the optimisation of game program parameters is a viable approach, especially if no good alternative exist for the types of parameters considered.

show abstract

Adaptive stochastic approximation by the simultaneous perturbation method

Cited by 326 publications

References 44 publications

Stochastic Optimization

Stochastic Optimization

Unbiased group-wise alignment by iterative central tendency estimations

Universal parameter optimisation in games based on SPSA

Contact Info

Product

Resources

About