Adaptive Incremental Gaussian Mixture Network for Non-Stationary Data Stream Classification

Chamby-Diaz, Jorge Cristhian; Recamonde‐Mendoza, Mariana; Bazzan, L. C. Ana; Grunitzki, and Ricardo

doi:10.1109/ijcnn.2018.8489049

Cited by 2 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…New components can have large overlapping regions with old components, interfering with what has been previously learned. Thus, we propose that, in future works, this issue must be addressed with a certain urgency, as this can deter practical use of the IGMN (the same issue was also found by [Koert et al 2018] and explored by [Chamby-Diaz et al 2018]).…”

Section: Discussionmentioning

confidence: 59%

“…This is necessary to avoid new components being created in overlapping positions with old ones, producing catastrophic forgetting. An alternative, which is explored in [Chamby-Diaz et al 2018], is to improve the stability of IGMN itself by avoiding overlaps automatically.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…Matrix inverses are computed at each step, something that is avoided in the present work by using the fast variant of the IGMN algorithm (FIGMN) Engel 2015, Pinto andEngel 2017]. An important contribution from the authors is the use of Q-value estimation variance provided by the GMM to drive exploration, resulting in something reminiscent of Q-value sampling in Bayesian Q-learning [Dearden et al 1998]. Another important insight is the need to forget old values, as Qvalues are non-stationary and learned through bootstrapping, meaning that old values are mostly wrong.…”

Section: Related Workmentioning

confidence: 99%

“…Also, since in most applications the number of outputs is much smaller than the number of inputs, this does not impose any huge penalty in complexity. Also note that the original FIGMN paper was missing the first two formulas, but they were derived and presented in [Chamby-Diaz et al 2018].…”

Section: Algorithm 3 Create Component Input: Xmentioning

confidence: 99%

See 3 more Smart Citations

Fast Reinforcement Learning with Incremental Gaussian Mixture Models

Pinto

2021

2021 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

This work presents a novel algorithm that integrates a data-efficient function approximator with reinforcement learning in continuous state spaces. An online and incremental algorithm capable of learning from a single pass through data, called Incremental Gaussian Mixture Network (IGMN), was employed as a sample-efficient function approximator for the joint state and Qvalues space, all in a single model, resulting in a concise and data-efficient algorithm, i.e., a reinforcement learning algorithm that learns from very few interactions with the environment. Results are analyzed to explain the properties of the obtained algorithm, and it is observed that the use of the IGMN function approximator brings some important advantages to reinforcement learning in relation to conventional neural networks trained by gradient descent methods.Recent works on episodic memory for reinforcement learning, such as Model-Free Episodic Control (MFEC) [Blundell et al. 2016] and Neural Episodic Control (NEC) , have introduced a framework for stable, long-term, and immediately accessible memories to be stored and used analogously to Q-tables. Those models allow for generalization through k-Nearest Neighbors (kNN) search or by the use of a Radial Basis Function (RBF) layer and a linear output layer. An issue with both methods is the rapid filling of the memory, which is dealt with by removing the least recently used entries. In [Agostinelli et al. 2019], authors propose to cluster the least recently used entry with its nearest entry, avoiding the removal of rare but important memories. That

show abstract

Section: Discussionmentioning

confidence: 59%

Section: Experiments and Resultsmentioning

confidence: 99%