A Constructive Incremental Learning Algorithm for Binary Classification Tasks

Giraud‐Carrier, Christophe; Martinez, Tony

doi:10.1109/smcals.2006.250718

Cited by 12 publications

(1 citation statement)

References 28 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In real-time sensing, inactive sensors or obsolete historical attributes are demanded to be dropped from the existing feature space (You et al 2018), or new attributes should be included since new sensors are put into work. For Internet applications like spam filtering or personalized news feeds, textual data are usually represented as a bag-of-words and the length of the feature vector is dy-namically changing (Giraud-Carrier 2000;Dekel, Shamir, and Xiao 2010).…”

Section: Introductionmentioning

confidence: 99%

When Online Learning Meets ODE: Learning without Forgetting on Variable Feature Space

2023

AAAI

View full text Add to dashboard Cite

Machine learning systems that built upon varying feature space are ubiquitous across the world. When the set of practical or virtual features changes, the online learning approach can adjust the learned model accordingly rather than re-training from scratch and has been an attractive area of research. Despite its importance, most studies for algorithms that are capable of handling online features have no ensurance of stationarity point convergence, while the accuracy guaranteed methods are still limited to some simple cases such as L_1 or L_2 norms with square loss. To address this challenging problem, we develop an efficient Dynamic Feature Learning System (DFLS) to perform online learning on the unfixed feature set for more general statistical models and demonstrate how DFLS opens up many new applications. We are the first to achieve accurate & reliable feature-wise online learning for a broad class of models like logistic regression, spline interpolation, group Lasso and Poisson regression. By utilizing DFLS, the updated model is theoretically the same as the model trained from scratch using the entire new feature space. Specifically, we reparameterize the feature-varying procedure and devise the corresponding ordinary differential equation (ODE) system to compute the optimal solutions of the new model status. Simulation studies reveal that the proposed DFLS can substantially ease the computational cost without forgetting.

show abstract

Section: Introductionmentioning

confidence: 99%