ScaIL: Classifier Weights Scaling for Class Incremental Learning

Belouadah, Eden; Popescu, Adrian

doi:10.1109/wacv45572.2020.9093562

Cited by 68 publications

(80 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The use of distillation loss is detrimental if exemplars per old class are allowed. A similar result is presented in ScaIL [45], and more experiments about the effect of distillation are analyzed. The authors made a hypothesis that the detriment is caused during the incremental learning process, and these errors are caused by class imbalance.…”

Section: B Main Componentssupporting

confidence: 57%

See 1 more Smart Citation

Incremental SAR Automatic Target Recognition With Error Correction and High Plasticity

Tang

Xiang

Zhang

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Synthetic aperture radar automatic target recognition (SAR ATR) uses computer processing capabilities to infer the classes of the targets without human intervention. For SAR ATR, deep learning gradually emerges as a powerful tool and achieves promising performance. However, it faces serious challenges of how to deal with incremental recognition scenarios. The existing deep learning-based SAR ATR methods usually predefine the total number of recognition classes. In realistic applications, the new tasks/classes will be added continuously. If all old data are stored and mixed with newly added data to update the model, the storage pressure and time consumption make the application infeasible. In this article, the high plastic error correction incremental learning (HPecIL) is proposed to address the model degradation and plasticity decline in the incremental scenario. Multiple optimal models trained on old tasks are used to correct accumulative errors and alleviate model degradation. Moreover, the sharp data distribution shift due to newly added data can also result in the model underperforming. A classbalanced training batch is constructed to deal with the issue of unbalanced data distribution. To make a trade-off between model stability and model plasticity, low-effect nodes in the model are removed to boost the efficiency of model update. The proposed HPecIL outperforms the other state-of-the-art methods in incremental recognition scenarios. The experimental results demonstrate the effectiveness of the proposed method.

show abstract

Section: B Main Componentssupporting

confidence: 57%

“…Most data replay-based incremental learning methods [22], [43], [44] follow the iCaRL experiment benchmark protocol to arrange classes and select exemplars. In ScaIL [45], the experimental results show that exemplar selection based on herding can improve performance.…”

Section: B Main Componentsmentioning

confidence: 99%

Incremental SAR Automatic Target Recognition With Error Correction and High Plasticity

Tang

Xiang

Zhang

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…In this sense, Incremental Learning with Dual Memory (LI2M) [47] corrects scores of old classes storing their statistical information in an additional memory. Classifier Weights Scaling for Class Incremental Learning (ScaIL) [51] rectifies the weights of old classes to make them more comparable to those of new classes. Zhao et al [36] proposed Weight Aligning (WA) to correct the biased weights at the output layer once the training process has ended.…”

Section: Related Workmentioning

confidence: 99%

A Class-Incremental Learning Method Based on Preserving the Learned Feature Space for EEG-Based Emotion Recognition

Jiménez-Guarneros

Alejo

2022

Mathematics

View full text Add to dashboard Cite

Deep learning-based models have shown to be one of the main active research topics in emotion recognition systems from Electroencephalogram (EEG) signals. However, a significant challenge is to effectively recognize new emotions that are incorporated sequentially, as current models must perform retraining from scratch. In this paper, we propose a Class-Incremental Learning (CIL) method, named Incremental Learning preserving the Learned Feature Space (IL2FS), in order to enable deep learning models to incorporate new emotions (classes) into the already known. IL2FS performs a weight aligning to correct the bias on new classes, while it incorporates margin ranking loss and triplet loss to preserve the inter-class separation and feature space alignment on known classes. We evaluated IL2FS over two public datasets (DREAMER and DEAP) for emotion recognition and compared it with other recent and popular CIL methods reported in computer vision. Experimental results show that IL2FS outperforms other CIL methods by obtaining an average accuracy of 59.08 ± 08.26% and 79.36 ± 04.68% on DREAMER and DEAP, recognizing data from new emotions that are incorporated sequentially.

show abstract

“…Therefore, there is a need for IL with a reasonable balance between accuracy, memory consumption, and training efficiency. In IoT, less or zero memory for historical data are preferred during the continuous evolving [11].…”

Section: Introductionmentioning

confidence: 99%

Class-Incremental Learning for Wireless Device Identification in IoT

Liu¹,

Wang²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

<div>Deep Learning (DL) has been utilized pervasively in the Internet of Things (IoT). One typical application of DL in IoT is device identification from wireless signals, namely Non-cryptographic Device Identification (NDI). However, learning components in NDI systems have to evolve to adapt to operational variations, such a paradigm is termed as Incremental Learning (IL). Various IL algorithms have been proposed and many of them require dedicated space to store the increasing amount of historical data, and therefore, they are not suitable for IoT or mobile applications. However, conventional IL schemes can not provide satisfying performance when historical data are not available. In this paper, we address the IL problem in NDI from a new perspective, firstly, we provide a new metric to measure the degree of topological maturity of DNN models from the degree of conflict of class-specific fingerprints. We discover that an important cause for performance degradation in IL enabled NDI is owing to the conflict of devices' fingerprints. Second, we also show that the conventional IL schemes can lead to low topological maturity of DNN models in NDI systems. Thirdly, we propose a new Channel Separation Enabled Incremental Learning (CSIL) scheme without using historical data, in which our strategy can automatically separate devices' fingerprints in different learning stages and avoid potential conflict. Finally, We evaluated the effectiveness of the proposed framework using real data from ADS-B (Automatic Dependent Surveillance-Broadcast), an application of IoT in aviation. The proposed framework has the potential to be applied to accurate identification of IoT devices in a variety of IoT applications and services. Data and code available at IEEE Dataport (DOI: 10.21227/1bxc-ke87) and \url{https://github.com/pcwhy/CSIL}}.<br></div>

show abstract

ScaIL: Classifier Weights Scaling for Class Incremental Learning

Cited by 68 publications

References 23 publications

Incremental SAR Automatic Target Recognition With Error Correction and High Plasticity

Incremental SAR Automatic Target Recognition With Error Correction and High Plasticity

A Class-Incremental Learning Method Based on Preserving the Learned Feature Space for EEG-Based Emotion Recognition

Class-Incremental Learning for Wireless Device Identification in IoT

Contact Info

Product

Resources

About