Possibi¬lities of Using Neural Network Incremental Learning

Abstract

The present time is characterized by unprecedented growth in the volume of information flows. Information processing underlies the solution of many practical problems. The intelligent infor-mation systems applications range is extremely extensive: from managing continuous technological processes in real-time to solving commercial and administrative problems. Intelligent information systems should have such a main property, as the ability to quickly process dynamical incoming da-ta in real-time. Also, intelligent information systems should be extracting knowledge from previously solved problems. Incremental neural network training has become one of the topical issues in ma-chine learning in recent years. Compared to traditional machine learning, incremental learning al-lows assimilating new knowledge that comes in gradually and preserving old knowledge gained from previous tasks. Such training should be useful in intelligent systems where data flows dynamically. Aim. Consider the concepts, problems, and methods of incremental neural network training, as well as assess the possibility of using it in intelligent systems development. Materials and methods. The idea of incremental learning, obtained in the analysis of a person's learning during his life, is consid-ered. The terms used in the literature to describe incremental learning are presented. The obstacles that arise in achieving the goal of incremental learning are described. A description of three scenari-os of incremental learning, among which class-incremental learning is distinguished, is given. An analysis of the methods of incremental learning, grouped into a family of techniques by the solution of the catastrophic forgetting problem, is given. The possibilities offered by incremental learning ver-sus traditional machine learning are presented. Results. The article attempts to assess the current state and the possibility of using incremental neural network learning, to identify differences from traditional machine learning. Conclusion. Incremental learning is useful for future intelligent sys-tems, as it allows to maintain existing knowledge in the process of updating, avoid learning from scratch, and dynamically adjust the model's ability to learn according to new data available.

Development and research of a neural network alternate incremental learning algorithm

“…В литературе термин «инкрементное обучение» (Incremental Learning) относится к инкрементальному росту, сокращению сети или онлайн-обучению. Также используются другие термины, такие как обучение на протяжении всей жизни, конструктивное обучение и эволюционное обучение, пошаговое и непрерывное обучение [3,4].…”