Combining offline and online classifiers for life-long learning

Fischer, Lydia; Hammer, Barbara; Wersing, Heiko

doi:10.1109/ijcnn.2015.7280678

Cited by 15 publications

(15 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently suggested strategies for continual learning include so-called Dedicated Memory Models and the appropriate combination of off-line and on-line learning [ 21 , 65 , 66 ]. Suitable rejection mechanisms for the mitigation of concept drift were recently considered in [ 67 ].…”

Section: Discussionmentioning

confidence: 99%

Statistical Mechanics of On-Line Learning Under Concept Drift

Straat

Abadi

Göpfert

et al. 2018

Entropy

Self Cite

View full text Add to dashboard Cite

We introduce a modeling framework for the investigation of on-line machine learning processes in non-stationary environments. We exemplify the approach in terms of two specific model situations: In the first, we consider the learning of a classification scheme from clustered data by means of prototype-based Learning Vector Quantization (LVQ). In the second, we study the training of layered neural networks with sigmoidal activations for the purpose of regression. In both cases, the target, i.e., the classification or regression scheme, is considered to change continuously while the system is trained from a stream of labeled data. We extend and apply methods borrowed from statistical physics which have been used frequently for the exact description of training dynamics in stationary environments. Extensions of the approach allow for the computation of typical learning curves in the presence of concept drift in a variety of model situations. First results are presented and discussed for stochastic drift processes in classification and regression problems. They indicate that LVQ is capable of tracking a classification scheme under drift to a non-trivial extent. Furthermore, we show that concept drift can cause the persistence of sub-optimal plateau states in gradient based training of layered neural networks for regression.

show abstract

Section: Discussionmentioning

confidence: 99%

Statistical Mechanics of On-Line Learning Under Concept Drift

Straat

Abadi

Göpfert

et al. 2018

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

“…In order to counter these problems, a learning algorithm is used to adapt to room changes at runtime. Therefore, a combination of batch learning and OL [8] based on LVQ is used to address the problem of the stability plasticity dilemma [9], [10], which describes the problem of a learning system to preserve acquired knowledge while at the same time new knowledge is being built up. In this offline online learning architecture (OOLA), basic knowledge is acquired to achieve stability in a batch learning process before runtime (offline).…”

Section: A Related Workmentioning

confidence: 99%

“…We still achieve about 90 % correct classifications on our fingerprinting environment data (Offline Setup Test Data, green and purple line), but the classification rates of the LTM for online data are too poor to be used in a real, changing environment. Fischer et al [8] emphasize the requirements of modern applications in the field of autonomous robots or driving systems for the use of incremental learning methods. They refer to the complexity of human learning strategies, which, despite the use of incremental methods, preserve basic concepts or basic knowledge, but can nevertheless quickly adapt to important changes.…”

Section: A Online Offline Memorymentioning

confidence: 99%

See 1 more Smart Citation

Online Offline Learning for Sound-Based Indoor Localization Using Low-Cost Hardware

et al. 2019

View full text Add to dashboard Cite

Online Learning algorithms and Indoor Positioning Systems are complex applications in the environment of cyber-physical systems. These distributed systems are created by networking intelligent machines and autonomous robots on the Internet of Things using embedded systems that enable the exchange of information at any time. This information is processed by Machine Learning algorithms to make decisions about current developments in production or to influence logistics processes for optimization purposes. In this article, we present and categorize the further development of the prototype of a novel Indoor Positioning System, which constantly adapts its knowledge to the conditions of its environment with the help of Online Learning. Here, we apply Online Learning algorithms in the field of sound-based indoor localization with low-cost hardware and demonstrate the improvement of the system over its predecessor and its adaptability for different applications in an experimental case study.

show abstract

“…Recently suggested strategies for continual learning include so-called Dedicated Memory Models and the appropriate combination of off-line and on-line learning [21,63,64]. Suitable rejection mechanisms for the mitigation of concept drift were recently considered in [65].…”

Section: Perspectives and Challengesmentioning

confidence: 99%

Statistical Mechanics of On-Line Learning Under Concept Drift

Straat¹,

Abadi

Göpfert

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

We introduce a modelling framework for the investigation of on-line machine learning processes in non-stationary environments. We exemplify the approach in terms of two specific model situations: In the first, we consider the learning of a classification scheme from clustered data by means of prototype-based Learning Vector Quantization (LVQ). In the second, we study the training of layered neural networks with sigmoidal activations for the purpose of regression. In both cases, the target, i.e. the classification or regression scheme, is considered to change continuously while the system is trained from a stream of labeled data. We extend and apply methods borrowed from statistical physics which have been used frequently for the exact description of training dynamics in stationary environments. Extensions of the approach allow for the computation of typical learning curves in the presence of concept drift in a variety of model situations. First results are presented and discussed for stochastic drift processes in classification and regression problems. They indicate that LVQ is capable of tracking a classification scheme under drift to a non-trivial extent. Furthermore, we show that concept drift can cause the persistence of sub-optimal plateau states in gradient based training of layered neural networks for regression.

show abstract

Combining offline and online classifiers for life-long learning

Cited by 15 publications

References 41 publications

Statistical Mechanics of On-Line Learning Under Concept Drift

Statistical Mechanics of On-Line Learning Under Concept Drift

Online Offline Learning for Sound-Based Indoor Localization Using Low-Cost Hardware

Statistical Mechanics of On-Line Learning Under Concept Drift

Contact Info

Product

Resources

About