Sequential randomized algorithms for sampled convex optimization

Sequential Randomized Algorithms for Convex Optimization in the Presence of Uncertainty

“…In this paper, which is an expanded version of [14], we study two new sequential algorithms for optimization, with full constraint satisfaction and partial constraint satisfaction, respectively, and we provide a rigorous analysis of their theoretical properties regarding the probability of violation of the returned solutions. These algorithms fall into the class of sequential probabilistic validation (SPV) algorithms introduced in [3].…”

A statistical learning theory approach for uncertain linear and bilinear matrix inequalities

“…Remark 3 (Comments on Algorithm 2 and related results) Algorithm 2 follows the general scheme of other sequential algorithms previously developed in the area of randomized algorithms for control of uncertain systems, see Calafiore et al (2011), and in particular Alamo et al (2012Alamo et al ( , 2009; Chamanbaz et al (2013b); Koltchinskii et al (2000). However, we remark that the sample bound M k in Algorithm 2 is strictly less conservative than the bound derived in Alamo et al (2012) because the infinite sum (Riemann Zeta function) is replaced with a finite sum, following ideas similar to those recently introduced in Chamanbaz et al (2013b). This enables us to choose α < 1 in (15) resulting in up to 30% improvement in the sample complexity.…”

“…Another important difference is on how the cardinality of the design sample set N k appears in the sequential algorithm. In (Chamanbaz et al, 2013b, Algorithm 1), the constraints are required to be satisfied for all the samples extracted from the set Q while, in Algorithm 2, we allow a limited number of samples to violate the constraints in (1) and ( 2), or their semidefinite versions, in both "design" and "validation" steps. Finally, we note that the sequential randomized algorithm in (Chamanbaz et al, 2013b, Algorithm 2) is purely based on additive and multiplicative Chernoff inequalities and hence may provide larger sample complexity than (15).…”

“…In (Chamanbaz et al, 2013b, Algorithm 1), the constraints are required to be satisfied for all the samples extracted from the set Q while, in Algorithm 2, we allow a limited number of samples to violate the constraints in (1) and ( 2), or their semidefinite versions, in both "design" and "validation" steps. Finally, we note that the sequential randomized algorithm in (Chamanbaz et al, 2013b, Algorithm 2) is purely based on additive and multiplicative Chernoff inequalities and hence may provide larger sample complexity than (15). It should also be remarked that the optimal values of the constants a and α depend on other parameters of the algorithm, such as the termination parameter k t , the accuracy level ε, and the level parameter ρ. Suboptimal values of a and α for which the sample bound ( 15) is minimized for a wide range of probabilistic levels are a = 3.05 and α = 0.9.…”

“…It should also be remarked that the optimal values of the constants a and α depend on other parameters of the algorithm, such as the termination parameter k t , the accuracy level ε, and the level parameter ρ. Suboptimal values of a and α for which the sample bound ( 15) is minimized for a wide range of probabilistic levels are a = 3.05 and α = 0.9. Note also that for ρ = 0 the optimal value of these parameters is a = ∞ and α = 0.1, and the bound (15) reduces to bound (12) in Chamanbaz et al (2013b). The parameter ρ plays a key role in the algorithm.…”