Stochastic approximation on discrete sets using simultaneous difference approximations

Abstract

Abstract-A stochastic approximation method for optimizing a class of discrete functions is considered. The procedure is a version of the Simultaneous Perturbation Stochastic Approximation (SPSA) method that has been modified to obtain a stochastic optimization method for cost functions defined on a grid of points in Euclidean p-space having integer components. We discuss the algorithm and examine its convergence properties.

Discrete simultaneous perturbation stochastic approximation on loss function with noisy measurements

“…Even though we describe the functions in continuous form, for DSPSA we only use their values at multivariate integer points. Strictly convex separable functions mentioned in corollary 1 are discussed in [10]. Then  * is also the optimal point of L. Because it is strictly convex, then for all    p \{ * } and any subgradient…”

Rate of convergence analysis of discrete simultaneous perturbation stochastic approximation algorithm

“…We further discuss it in Lemma 1 below. 4. Condition (vi) is used to keep the stability of the algorithm and this condition is automatically satisfied when k is large enough.…”

“…In this paper, we discuss the rate of convergence of DSPSA. We show that k P = θ θ can be used to compare DSPSA with other discrete stochastic algorithms (Alrefaei and Andradottir (1999), Andradottir (1995), Yan and Mukai (1992), Gong, Ho and Zhai (2000), Hong and Nelson (2006), Sklenar and Popela (2010), Hill (2004), Spall (2003) etc).…”

Discrete Convexity and Stochastic Approximation for Cross-layer On– off Transmission Control

“…1 On the other hand, if one can prove the convexity of the objective function, these conditions are usually straightforwardly satisfied. In addition, there exists SA algorithms that are exclusively proposed for discrete convex minimization problems in the literature for which the global and almost sure convergence is guaranteed, e.g., [22]. The other problem with the SA algorithms in [14], [16], [17] is that CSPSA is originally proposed for continuous minimization problems.…”