On the Solution of Stochastic Optimization and Variational Problems in Imperfect Information Regimes

Abstract

Abstract. We consider the solution of a stochastic convex optimization problem E[f (x; θ * , ξ)] over a closed and convex set X in a regime where θ * is unavailable and ξ is a suitably defined random variable. Instead, θ * may be obtained through the solution of a learning problem that requires minimizing a metric E[g(θ; η)] in θ over a closed and convex set Θ. Traditional approaches have been either sequential or direct variational approaches. In the case of the former, this entails the following steps: (i) a solution to the learning problem, namely θ * , is obtained; and (ii) a solution is obtained to the associated computational problem which is parametrized by θ * . Such avenues prove difficult to adopt particularly since the learning process has to be terminated finitely and consequently, in large-scale instances, sequential approaches may often be corrupted by error. On the other hand, a variational approach requires that the problem may be recast as a possibly non-monotone stochastic variational inequality problem in the (x, θ) space; but there are no known first-order stochastic approximation schemes are currently available for the solution of this problem. To resolve the absence of convergent efficient schemes, we present a coupled stochastic approximation scheme which simultaneously solves both the computational and the learning problems. The obtained schemes are shown to be equipped with almost sure convergence properties in regimes when the function f is either strongly convex as well as merely convex. Importantly, the scheme displays the optimal rate for strongly convex problems while in merely convex regimes, through an averaging approach, we quantify the degradation associated with learning by noting that the error in function value after K steps is O ln(K)/K , rather than O 1/K when θ * is available. Notably, when the averaging window is modified suitably, it can be see that the originakl rate of O 1/K is recovered. Additionally, we consider an online counterpart of the misspecified optimization problem and provide a non-asymptotic bound on the average regret with respect to an offline counterpart. In the second part of the paper, we extend these statements to a class of stochastic variational inequality problems, an object that unifies stochastic convex optimization problems and a range of stochastic equilibrium problems. Analogous almost-sure convergence statements are provided in strongly monotone and merely monotone regimes, the latter facilitated by using an iterative Tikhonov regularization. In the merely monotone regime, under a weak-sharpness requirement, we quantify the degradation associated with learning and show that expected error associated with dist(x k , X * ) is O ln(K)/K . Preliminary numerics demonstrate the performance of the prescribed schemes.

Exploiting problem structure in optimization under uncertainty via online convex optimization

“…Jiang and Shanbhag [21,22] introduced and studied the JEO problem (Opt(u * ))-(Est) in a stochastic setting, and Ahmadi and Shanbhag [2] examined the deterministic case. In this paper, we consider the deterministic JEO problem, for which [2] provided some remarkable convergence results.…”

“…The main problem is that at each step t, the information from the previous steps cannot be utilized, hence they are essentially wasted. To address this, Jiang and Shanbhag [21,22] and Ahmadi and Shanbhag [2] propose a scheme that jointly solves the estimation and optimization problems, which we refer to as JEO. With this scheme, they can efficiently generate a sequence of points x t and u t such that f (x t , u t ) will indeed converge to the desired minimum (Opt(u * )), and give corresponding non-asymptotic error rates.…”

“…These new developments then allow us to partially resolve an open question from [5] on the complexity lower bounds for solving RO via iterative techniques. For JEO, in addition to covering the standard setups from [2] in a unified manner and extending them to the more general proximal setup, we explore a setting which was not covered in the work of [21,22,2], when f is non-smooth and strongly convex. In this setting, we provide an improved convergence rate of O(1/T ), which is the optimal rate even if we had the correct data u * upfront.…”

Algorithms for stochastic optimization with function or expectation constraints

“…Moreover, the rate of convergence of these methods for functional constrained problems has not been well-understood other than conic constraints even for the deterministic setting. Third, in [16] (and see references therein), Jiang and Shanbhag developed a coupled SA method to solve a stochastic optimization problem with parameters given by another optimization problem, and hence is not applicable to problem (1.4)-(1.5). Moreover, each iteration of their method requires two stochastic subgradient projection steps and hence is more expensive than that of CSPA.…”

Optimal Learning for Stochastic Optimization with Nonlinear Parametric Belief Models

“…To the best of our knowledge, this is the first optimal learning paper that addresses the dual problems of objective maximization with parameter identification for problems with parametric belief models (other literatures with similar dual-objective formulations usually assume that experiments are inexpensive, e.g, [19]). Previous papers only concentrate on discovering the optimal alternative, but in many real world situations, scientists also care about getting accurate estimates of the parameters.…”