Two-time scale learning automata: an efficient decision making mechanism for stochastic nonlinear resource allocation

Abstract

The Stochastic Non-linear Fractional Equality Knapsack (NFEK) prob lem is a substantial resource allocation problem which admits a large set of applications such as web polling under polling constraints, and constrained estimation. The NFEK problem is usually solved by trial and error based on noisy feedback information from the environment. The available solutions to NFEK are based on the traditional family of Reward-Inaction Learning Au tomata (LA) scheme where the action probabilities are updated based on only the last feedback. Such an update form seems counterproductive for two rea sons: 1) it only uses the last feedback and does not consider the whole history of the feedback and 2) it ignores updates whenever the last feedback does not correspond to a reward. In this pape1� we rather suggest instead a learn ing solution that resorts to the whole history of feedback using the theory of two time-scale separation. Through comprehensive experimental results we show that the proposed solution is not only superior to the state-of-the-art in terms of peak performance but is also robust to the choice of the tuning parameters.

Adaptive Learning with Artificial Barriers Yielding Nash Equilibria in General Games

“…Applications of LA: LA had been utilized in many applications over the years. Recent applications of LA include achieving fair load balancing based on two-time-scale separation paradigm [10], detection of malicious social bots [11], secure socket layer certificate verification [12], a protocol for intrusion detection [13], efficient decision making mechanism for stochastic nonlinear resource allocation [14], link prediction in stochastic social networks [15], user behavior analysisbased smart energy management for webpage ranking [16], and resource selection in computational grids [17], to mention a few. LA Applied to Game Theory: Studies on strategic games with LA were focused mainly on traditional L R−I which is desirable to use as it can yield Nash equilibrium in pure strategies [2].…”