Implementation of Incremental Learning in Artificial Neural Networks

Andrade, M. de; Gasca, Eduardo; Rendón, Eréndira

doi:10.29007/8559

Cited by 7 publications

(6 citation statements)

References 11 publications

(12 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As the number of classes grows, this requirement becomes infeasible. Incremental learning is a learning method where the learning process takes place when new classes emerge and the existing knowledge is adjusted based on what has been newly learnt (Andrade et al, 2017). The entire training set is not needed before the learning process, and the new classes can appear over time.…”

Section: Need For Incremental Learningmentioning

confidence: 99%

RETRACTED: Bayesian approach to incremental batch learning on forest cover sensor data for multiclass classification

D¹,

Venkataramana²,

Saraswathi³

et al. 2021

Concurrent Engineering

View full text Add to dashboard Cite

Deep neural networks can be used to perform nonlinear operations at multiple levels, such as a neural network that is composed of many hidden layers. Although deep learning approaches show good results, they have a drawback called catastrophic forgetting, which is a reduction in performance when a new class is added. Incremental learning is a learning method where existing knowledge should be retained even when new data is acquired. It involves learning with multiple batches of training data and the newer learning sessions do not require the data used in the previous iterations. The Bayesian approach to incremental learning uses the concept of the probability distribution of weights. The key idea of Bayes theorem is to find an updated distribution of weights and biases. In the Bayesian framework, the beliefs can be updated iteratively as the new data comes in. Bayesian framework allows to update the beliefs iteratively in real-time as data comes in. The Bayesian model for incremental learning showed an accuracy of 82%. The execution time for the Bayesian model was lesser on GPU (670 s) when compared to CPU (1165 s).

show abstract

Section: Need For Incremental Learningmentioning

confidence: 99%

RETRACTED: Bayesian approach to incremental batch learning on forest cover sensor data for multiclass classification

D¹,

Venkataramana²,

Saraswathi³

et al. 2021

Concurrent Engineering

View full text Add to dashboard Cite

show abstract

“…adjustment. Andrade et al (2017), Castro et al (2018, and Rusu et al (2016) investigate adapting neural networks to new data with additional classes or even new tasks, requiring to change the structure of the neural network. Our setting is less complex as the neural network is trained on all possible classes from the beginning.…”

Section: Continuous Model Adjustmentmentioning

confidence: 99%

Analysis of Automatic Annotation Suggestions for Hard Discourse-Level Tasks in Expert Domains

Schulz

Meyer

Kiesewetter

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Many complex discourse-level tasks can aid domain experts in their work but require costly expert annotations for data creation. To speed up and ease annotations, we investigate the viability of automatically generated annotation suggestions for such tasks. As an example, we choose a task that is particularly hard for both humans and machines: the segmentation and classification of epistemic activities in diagnostic reasoning texts. We create and publish a new dataset covering two domains and carefully analyse the suggested annotations. We find that suggestions have positive effects on annotation speed and performance, while not introducing noteworthy biases. Envisioning suggestion models that improve with newly annotated texts, we contrast methods for continuous model adjustment and suggest the most effective setup for suggestions in future expert tasks.

show abstract

“…The incremental learning capability of the ANN grants keeping the information acquired at the beginning of the search and enhances the surrogate quality over the new regions of interest [49].…”

Section: Surrogate Building Using Mcdropout-based Bnnmentioning

confidence: 99%

Parallel surrogate-assisted optimization: Batched Bayesian Neural Network-assisted GA versus q-EGO

Briffoteaux

Gobert

Ragonnet

et al. 2020

Swarm and Evolutionary Computation

View full text Add to dashboard Cite

Surrogate-based optimization is widely used to deal with long-running black-box simulation-based objective functions. Actually, the use of a surrogate model such as Kriging or Artificial Neural Network allows to reduce the number of calls to the CPU time-intensive simulator. Bayesian optimization uses the ability of surrogates to provide useful information to help guiding effectively the optimization process. In this paper, the Efficient Global Optimization (EGO) reference framework is challenged by a Bayesian Neural Network-assisted Genetic Algorithm, namely BNN-GA. The Bayesian Neural Network (BNN) surrogate is chosen for its ability to provide an uncertainty measure of the prediction that allows to compute the Expected Improvement of a candidate solution in order to improve the exploration of the objective space. BNN is also more reliable than Kriging models for high-dimensional problems and faster to set up thanks to its incremental training. In addition, we propose a batch-based approach for the parallelization of BNN-GA that is challenged by a parallel version of EGO, called q-EGO. Parallel computing is a highly important complementary way (to surrogates) to deal with the computational burden of simulation-based optimization. The comparison of the two parallel approaches is experimentally performed through several benchmark functions and two real-world problems within the scope of Tuberculosis Transmission Control (TBTC). The study presented in this paper proves that parallel batched BNN-GA is a viable alternative to q-EGO approaches being more suitable for high-dimensional problems, parallelization impact, bigger databases and moderate search budgets. Moreover, a significant improvement of the solutions is obtained for the two TBTC problems tackled.

show abstract

Implementation of Incremental Learning in Artificial Neural Networks

Cited by 7 publications

References 11 publications

RETRACTED: Bayesian approach to incremental batch learning on forest cover sensor data for multiclass classification

RETRACTED: Bayesian approach to incremental batch learning on forest cover sensor data for multiclass classification

Analysis of Automatic Annotation Suggestions for Hard Discourse-Level Tasks in Expert Domains

Parallel surrogate-assisted optimization: Batched Bayesian Neural Network-assisted GA versus q-EGO

Contact Info

Product

Resources

About