On the power of incremental learning

Lange, Steffen; Grieser, Gunter

doi:10.1016/s0304-3975(01)00404-2

Cited by 24 publications

(12 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Generally speaking, data stream is a sequence of unbounded, real-time data items with a very high rate that can be read only once by an application (Gaber et al 2003). The restriction placed by the end of this definition is also called one-pass constraint (Aggarwal 2007), which is also claimed by other literature (Sharma 1998;Lange and Grieser 2002;Muhlbaier et al 2009). It has been flourished for quite a few years for the studies of learning from data stream.…”

Section: Introductionmentioning

confidence: 85%

Towards incremental learning of nonstationary imbalanced data stream: a multiple selectively recursive approach

Chen

2010

Evolving Systems

124

View full text Add to dashboard Cite

Difficulties of learning from nonstationary data stream are generally twofold. First, dynamically structured learning framework is required to catch up with the evolution of unstable class concepts, i.e., concept drifts. Second, imbalanced class distribution over data stream demands a mechanism to intensify the underrepresented class concepts for improved overall performance. To alleviate the challenges brought by these issues, we propose the recursive ensemble approach (REA) in this paper. To battle against the imbalanced learning problem in training data chunk received at any timestamp t, i.e., S t ; REA adaptively pushes into S t part of minority class examples received within [0, t -1] to balance its skewed class distribution. Hypotheses are then progressively developed over time for all balanced training data chunks and combined together as an ensemble classifier in a dynamically weighted manner, which therefore addresses the concept drifts issue in time. Theoretical analysis proves that REA can provide less erroneous prediction results than a comparative algorithm. Besides that, empirical study on both synthetic benchmarks and real-world data set is also applied to validate effectiveness of REA as compared with other algorithms in terms of evaluation metrics consisting of overall prediction accuracy and ROC curve.

show abstract

Section: Introductionmentioning

confidence: 85%

Towards incremental learning of nonstationary imbalanced data stream: a multiple selectively recursive approach

Chen

2010

Evolving Systems

124

View full text Add to dashboard Cite

show abstract

“…Therefore, several alternative approaches to incremental learning have been developed, including online learning algorithms that learn one instance at a time [16], [17], and partial memory and boundary analysis algorithms that memorize a carefully selected subset of extreme examples that lie along the decision boundaries [18]- [22]. However, such algorithms have limited applicability for realworld NDE problems due to restrictions on classifier type, number of classes that can be learned, or the amount of data that can be analyzed.…”

Section: A Incremental Learningmentioning

confidence: 99%

An incremental learning algorithm with confidence estimation for automated identification of NDE signals

Polikar

Удпа

et al. 2004

IEEE Trans. Ultrason., Ferroelect., Freq. Contr.

View full text Add to dashboard Cite

Abstract-An incremental learning algorithm is introduced for learning new information from additional data that may later become available, after a classifier has already been trained using a previously available database. The proposed algorithm is capable of incrementally learning new information without forgetting previously acquired knowledge and without requiring access to the original database, even when new data include examples of previously unseen classes. Scenarios requiring such a learning algorithm are encountered often in nondestructive evaluation (NDE) in which large volumes of data are collected in batches over a period of time, and new defect types may become available in subsequent databases. The algorithm, named Learn++, takes advantage of synergistic generalization performance of an ensemble of classifiers in which each classifier is trained with a strategically chosen subset of the training databases that subsequently become available. The ensemble of classifiers then is combined through a weighted majority voting procedure. Learn++ is independent of the specific classifier(s) comprising the ensemble, and hence may be used with any supervised learning algorithm. The voting procedure also allows Learn++ to estimate the confidence in its own decision. We present the algorithm and its promising results on two separate ultrasonic weld inspection applications.

show abstract

“…While a variety of definitions for incremental learning exist in the literature, we propose a general definition due to Muhlbaier et al [67], outlined by several authors [32,37,55,56]. Namely, a learning algorithm is incremental if, for a sequence of training instances (potentially batches of instances), it satisfies the following criteria:…”

Section: Introductionmentioning

confidence: 99%

Learning from streaming data with concept drift and imbalance: an overview

2012

View full text Add to dashboard Cite

The primary focus of machine learning has traditionally been on learning from data assumed to be sufficient and representative of the underlying fixed, yet unknown, distribution. Such restrictions on the problem domain paved the way for development of elegant algorithms with theoretically provable performance guarantees. As is often the case, however, real-world problems rarely fit neatly into such restricted models. For instance class distributions are often skewed, resulting in the "class imbalance" problem. Data drawn from non-stationary distributions is also common in real-world applications, resulting in the "concept drift" or "non-stationary learning" problem which is often associated with streaming data scenarios. Recently, these problems have independently experienced increased research attention, however, the combined problem of addressing all of the above mentioned issues has enjoyed relatively little research. If the ultimate goal of intelligent machine learning algorithms is to be able to address a wide spectrum of real-world scenarios, then the need for a general framework for learning from, and adapting to, a non-stationary environment that may introduce imbalanced data can be hardly overstated. In this paper, we first present an overview of each of these challenging areas, followed by a comprehensive review of recent research for developing such a general framework.

show abstract

On the power of incremental learning

Cited by 24 publications

References 19 publications

Towards incremental learning of nonstationary imbalanced data stream: a multiple selectively recursive approach

Towards incremental learning of nonstationary imbalanced data stream: a multiple selectively recursive approach

An incremental learning algorithm with confidence estimation for automated identification of NDE signals

Learning from streaming data with concept drift and imbalance: an overview

Contact Info

Product

Resources

About