Asynchronous action-reward learning for nonstationary serial supply chain inventory control

Kim, Chang Ouk; Kwon, Ick-Hyun; Baek, Jun-Geol

doi:10.1007/s10489-007-0038-2

Cited by 21 publications

(1 citation statement)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They have been used in game playing [4][5][6][7][8][9][10], parameter optimization [11,12], channel selection in cognitive radio networks [13], assigning capacities in prioritized networks [14], solving knapsack problems [15], optimizing the web polling problem [16,17], stochastically optimally allocating limited resources [15,18,19], service selection in stochastic environments [20], numerical optimization [21], web crawling [22], microassembly path planning [23], multiagent learning [24], and in batch sequencing and sizing in just-intime manufacturing systems [25]. An asynchronous actionreward learning has been used for nonstationary serial supply chain inventory control [26].…”

Section: Learning Automata: Concept and Their Applicationsmentioning

confidence: 99%

On incorporating the paradigms of discretization and Bayesian estimation to create a new family of pursuit learning automata

2013

View full text Add to dashboard Cite

There are currently two fundamental paradigms that have been used to enhance the convergence speed of Learning Automata (LA). The first involves the concept of utilizing the estimates of the reward probabilities, while the second involves discretizing the probability space in which the LA operates. This paper demonstrates how both of these can be simultaneously utilized, and in particular, by using the family of Bayesian estimates that have been proven to have distinct advantages over their maximum likelihood counterparts. The success of LA-based estimator algorithms over the classical, Linear Reward-Inaction (L RI )-like schemes, can be explained by their ability to pursue the actions with the highest reward probability estimates. Without access to reward probability estimates, it makes sense for schemes like the L RI to first make large exploring steps, and then to gradually turn exploration into exploitation by making progressively smaller learning steps. However, this behavior becomes counter-intuitive when pur- suing actions based on their estimated reward probabilities. Learning should then ideally proceed in progressively larger steps, as the reward probability estimates turn more accurate. This paper introduces a new estimator algorithm, the Discretized Bayesian Pursuit Algorithm (DBPA), that achieves this by incorporating both the above paradigms. The DBPA is implemented by linearly discretizing the action probability space of the Bayesian Pursuit Algorithm (BPA) (Zhang et al. in IEA-AIE 2011, Springer, New York, pp. 608-620, 2011. The key innovation of this paper is that the linear discrete updating rules mitigate the counterintuitive behavior of the corresponding linear continuous updating rules, by augmenting them with the reward probability estimates. Extensive experimental results show the superiority of DBPA over previous estimator algorithms. Indeed, the DBPA is probably the fastest reported LA to date. Apart from the rigorous experimental demonstration of the strength of the DBPA, the paper also briefly records the proofs of why the BPA and the DBPA are -optimal in stationary environments.

show abstract

Section: Learning Automata: Concept and Their Applicationsmentioning

confidence: 99%

On incorporating the paradigms of discretization and Bayesian estimation to create a new family of pursuit learning automata

2013

View full text Add to dashboard Cite

show abstract

Machine Learning Applications in the Supply Chain, a Literature Review

Rosenberg-Vitorica

Salais-Fierro²,

Marmolejo-Saucedo³

et al. 2023

Engineering Cyber-Physical Systems and Critical Infrastructures

View full text Add to dashboard Cite

Intelligent Inventory Control: Is Bootstrapping Worth Implementing?

Katanyukul

Chong

Duff

2012

Intelligent Information Processing VI

View full text Add to dashboard Cite

The common belief is that using Reinforcement Learning methods (RL) with bootstrapping gives better results than without. However, inclusion of bootstrapping increases the complexity of the RL implementation and requires significant effort. This study investigates whether inclusion of bootstrapping is worth the effort when applying RL to inventory problems. Specifically, we investigate bootstrapping of the temporal difference learning method by using eligibility trace. In addition, we develop a new bootstrapping extension to the Residual Gradient method to supplement our investigation. The results show questionable benefit of bootstrapping when applied to inventory problems. Significance tests could not confirm that bootstrapping had statistically significantly reduced costs of inventory controlled by a RL agent. Our empirical results are based on a variety of problem settings, including demand correlations, demand variances, and cost structures.

show abstract

Asynchronous action-reward learning for nonstationary serial supply chain inventory control

Cited by 21 publications

References 24 publications

On incorporating the paradigms of discretization and Bayesian estimation to create a new family of pursuit learning automata

On incorporating the paradigms of discretization and Bayesian estimation to create a new family of pursuit learning automata

Machine Learning Applications in the Supply Chain, a Literature Review

Intelligent Inventory Control: Is Bootstrapping Worth Implementing?

Contact Info

Product

Resources

About