Continual prune-and-select: class-incremental learning with specialized subnetworks

Abstract

We introduce a new continual (or lifelong) learning algorithm called LDA-CP&S that performs segmentation tasks without undergoing catastrophic forgetting. The method is applied to two different surface defect segmentation problems that are learned incrementally, i.e. providing data about one type of defect at a time, while still being capable of predicting every defect that was seen previously. Our method creates a defect-related subnetwork for each defect type via iterative pruning and trains a classifier based on linear discriminant analysis (LDA). At the inference stage, we first predict the defect type with LDA and then predict the surface defects using the selected subnetwork. We compare our method with other continual learning methods showing a significant improvement -mean Intersection over Union better by a factor of two when compared to existing methods on both datasets. Importantly, our approach shows comparable results with joint training when all the training data (all defects) are seen simultaneously * .

Visually Grounded Continual Language Learning with Selective Specialization

“…Introducing task-specific parameters. Research on continual neural pruning assigns some model capacity to each task by iteratively pruning and retraining (sub-)networks that are specialized to each task (Mallya and Lazebnik, 2018;Geng et al, 2021;Dekhovich et al, 2023;Kang et al, 2022;Hung et al, 2019;Gurbuz and Dovrolis, 2022;Jung et al, 2020;Wang et al, 2022). However, while such methods are effective in overcoming forgetting, the evolution and learning behavior of pruned subnetworks provides little interpretability regarding the role of individual parts of the network in solving CL problems.…”

A multifidelity approach to continual learning for physical systems

“…Recent work has looked at improving the training of PINNs for such systems, including applications of the neural tangent kernel [33], but more work remains to be done. The closest work we are aware of for continual learning with PINNS is the backward-compatible PINNs in [34] and incremental PINNs (iPINNs) in [35]. Backward-compatible PINNs train N PINNs on a sequence of N time domains, and in each new domain enforce that the output from the current PINN satisfies the PINN loss function in the current domain and the output of the previous model on all previous domains.…”

DRAMA: Joint Risk Localization and Captioning in Driving

“…With its need for finetuning on a series of datasets, this approach does not alleviate the ongoing concern of substantial computational costs for model execution. Contrary to this trend, few research efforts (Dekhovich et al, 2023) have been recently made by applying pruning methods within a continual learning context. However, they still require re-training of the base pre-trained models on the new datasets, with a potential reduction of their generalizability originially set toward various tasks.…”