Prevention of catastrophic interference and imposing active forgetting with generative methods

Sukhov, Sergey; Leontev, Mikhail Iu.; Miheev, Alexander; Sviatov, Kirill

doi:10.1016/j.neucom.2020.03.024

Cited by 13 publications

(6 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If we compare the local learning of the radial basis function network with the global learning technique of the feed-forward neural network, the latter suffers from catastrophic forgetting. Kirkpatrick et al (2017) and Sukhov et al (2020) look at ways of improving this issue, specifically at training networks that can maintain expertise on tasks that they have not experienced for a long time. The radial basis function network that we formulate, is naturally designed to measure the similarity between test samples and continuously updated prototypes that capture the characteristics of the feature space.…”

Section: Discussionmentioning

confidence: 99%

Online Learning with Radial Basis Function Networks

Borrageiro¹,

Firoozye²,

Barucca³

2022

JFDS

View full text Add to dashboard Cite

We investigate the benefits of feature selection, nonlinear modelling and online learning when forecasting in financial time series. We consider the sequential and continual learning sub-genres of online learning. The experiments we conduct show that there is a benefit to online transfer learning, in the form of radial basis function networks, beyond the sequential updating of recursive least-squares models. We show that the radial basis function networks, which make use of clustering algorithms to construct a kernel Gram matrix, are more beneficial than treating each training vector as separate basis functions, as occurs with kernel Ridge regression. We demonstrate quantitative procedures to determine the very structure of the radial basis function networks. Finally, we conduct experiments on the log returns of financial time series and show that the online learning models, particularly the radial basis function networks, are able to outperform a random walk baseline, whereas the offline learning models struggle to do so.

show abstract

Section: Discussionmentioning

confidence: 99%

Online Learning with Radial Basis Function Networks

Borrageiro¹,

Firoozye²,

Barucca³

2022

JFDS

View full text Add to dashboard Cite

show abstract

“…If we compare the local learning of the radial basis function network with the global learning technique of the feed-forward neural network, the latter suffers from catastrophic forgetting. Kirkpatrick et al (2017) and Sukhov et al (2020) look at ways of improving this issue, specifically at training networks that can maintain expertise on tasks that they have not experienced for a long time. The radial basis function networks that we formulate are naturally designed to measure the similarity between test samples and continuously updated prototypes that capture the characteristics of the feature space.…”

Section: Discussionmentioning

confidence: 99%

Online Learning with Radial Basis Function Networks

Borrageiro¹

2021

Preprint

View full text Add to dashboard Cite

We investigate the benefits of feature selection, nonlinear modelling and online learning when forecasting financial time series. We combine sequential updating with continual learning, specifically transfer learning. We perform feature representation transfer through clustering algorithms that determine the analytical structure of radial basis function networks we construct. These networks achieve lower mean-square prediction errors than kernel ridge regression models, which arbitrarily use all training vectors as prototypes. We also demonstrate quantitative procedures to determine the very structure of the networks. Finally, we conduct experiments on the log-returns of financial time series and show that these online transfer learning models outperform a random-walk baseline. In contrast, the offline learning models struggle to do so.

show abstract

“…However, with increasing runtime of the framework, the amount of data collected from the use case increases. This leads to large data sets that do not necessarily contribute to good performance of the overall system, as the information may become outdated [60], [61]. Hence, it is useful to develop a strategy on how to discard or aggregate the increasing amount of data.…”

Section: Discussionmentioning

confidence: 99%

A Case Study on Optimization of Platooning Coordination

Hadry¹,

Kounev²,

Krupitzer³

2021

Preprint

View full text Add to dashboard Cite

In today's world, circumstances, processes, and requirements for software systems are becoming increasingly complex. In order to operate properly in such dynamic environments, software systems must adapt to these changes, which has led to the research area of Self-Adaptive Systems (SAS). Platooning is one example of adaptive systems in Intelligent Transportation Systems, which is the ability of vehicles to travel with close intervehicle distances. This technology leads to an increase in road throughput and safety, which directly addresses the increased infrastructure needs due to increased traffic on the roads. However, the No-Free-Lunch theorem states that the performance of one platooning coordination strategy is not necessarily transferable to other problems. Moreover, especially in the field of SAS, the selection of the most appropriate strategy depends on the current situation of the system. In this paper, we address the problem of self-aware optimization of adaptation planning strategies by designing a framework that includes situation detection, strategy selection, and parameter optimization of the selected strategies. We apply our approach on the case study platooning coordination and evaluate the performance of the proposed framework.

show abstract

Prevention of catastrophic interference and imposing active forgetting with generative methods

Cited by 13 publications

References 66 publications

Online Learning with Radial Basis Function Networks

Online Learning with Radial Basis Function Networks

Online Learning with Radial Basis Function Networks

A Case Study on Optimization of Platooning Coordination

Contact Info

Product

Resources

About