Stochastic optimization over continuous and discrete variables with applications to concept learning under noise

Abstract

We consider optimization problems where the objective function is defined over some continuous and some discrete variables, and only noise corrupted values of the objective function are observable. Such optimization problems occur naturally in PAC learning with noisy samples. We propose a stochastic learning algorithm based on the model of a hybrid team of learning automata involved in a stochastic game with incomplete information to solve this optimization problem and establish its convergence properties. We then illustrate an application of this automata model in learning a class of conjunctive logic expressions over both nominal and linear attributes under noise.

Varieties of learning automata: an overview

“…We can define optimal points of the game to be those which are like Nash equilibria with respect to the discrete part and local maxima (in the Euclidean sense) with respect to the continuous part. It can be shown that if each of the FALA uses algorithm and each of the CALA uses the algorithm given earlier in this section, then the team would converge to one of the optimal points [38]. Such optimization problems, where the objective function is defined over some discrete and some continuous variables, are useful in applications such as learning concepts in the form of logic expressions [38], [39].…”

“…As illustrated briefly in Section VII, automata algorithms have been used for learning rich classes of pattern classifiers [42]. Due to the fact that the action-sets of automata need not have any algebraic structure on them, similar algorithms are seen to be useful for concept learning [38], [39]. LA models have also been used successfully for many problems involving adaptive decision making in communication systems.…”

Two Timescale Analysis of the Alopex Algorithm for Optimization

“…Another set of closely related methods are the reinforcement learning techniques (Narendra & Thathachar, 1989;Barto & Jordan, 1987;Williams, 1992;Santharam, Sastry, & Thathachar, 1994;Rajaraman & Sastry, 1999). These algorithms do not explicitly estimate the gradient; instead, they make stochastic moves in the parameter space using some distributions (which may be adapted at each iteration) so as to follow the gradient in an expected sense.…”

A new continuous action-set learning automaton for function optimization

“…This CALA is used for stochastic optimization and needs at most two function evaluations in each iteration, irrespective of the dimension of the parameter space. In [11], a team of FALA and CALA is also used for stochastic optimization. In [12], a CALA, which is called continuous action reinforcement learning automata (CARLA) is given for adaptive control, but no formal study is conducted to explain its behavior.…”