Class-Incremental Learning for Semantic Segmentation Re-Using Neither Old Data Nor Old Labels

Klingner, Marvin; Bär, Andreas; Donn, Philipp; Fingscheidt, Tim

doi:10.1109/itsc45102.2020.9294483

Cited by 37 publications

(51 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Continual Learning (CL) is a more mature field where previous knowledge acquired on a subset of data is further expanded to learn new tasks (e.g., class labels) [19,20,21]. Continual learning has recently been investigated also in semantic segmentation [3,22,23,24,25,26]. The most popular strategy to retain knowledge from the previous learning step is the usage of knowledge distillation constraints at either the output [3,23,24,22] or feature level [3,25].…”

Section: Related Workmentioning

confidence: 99%

“…Continual learning has recently been investigated also in semantic segmentation [3,22,23,24,25,26]. The most popular strategy to retain knowledge from the previous learning step is the usage of knowledge distillation constraints at either the output [3,23,24,22] or feature level [3,25]. Another promising research direction involves feature-level regularization to increase separation among features from different classes [23].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Continual Coarse-to-Fine Domain Adaptation in Semantic Segmentation

Shenaj,

Barbato,

Michieli

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep neural networks are typically trained in a single shot for a specific task and data distribution, but in real world settings both the task and the domain of application can change. The problem becomes even more challenging in dense predictive tasks, such as semantic segmentation, and furthermore most approaches tackle the two problems separately. In this paper we introduce the novel task of coarse-to-fine learning of semantic segmentation architectures in presence of domain shift. We consider subsequent learning stages progressively refining the task at the semantic level; i.e., the finer set of semantic labels at each learning step is hierarchically derived from the coarser set of the previous step. We propose a new approach (CCDA) to tackle this scenario. First, we employ the maximum squares loss to align source and target domains and, at the same time, to balance the gradients between well-classified and harder samples. Second, we introduce a novel coarse-to-fine knowledge distillation constraint to transfer network capabilities acquired on a coarser set of labels to a set of finer labels. Finally, we design a coarse-to-fine weight initialization rule to spread the importance from each coarse class to the respective finer classes. To evaluate our approach, we design two benchmarks where source knowledge is extracted from the GTA5 dataset and it is transferred to either the Cityscapes or the IDD datasets, and we show how it outperforms the main competitors.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Continual Coarse-to-Fine Domain Adaptation in Semantic Segmentation

Shenaj,

Barbato,

Michieli

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Klingner et al [114] note that the existing approaches are either restricted to settings in which the additional classes have no overlap with the old ones or rely on labels for both old and new classes. The authors introduce a generally applicable technique that learns new data solely from labels for the new classes and outputs of a pretrained teacher model.…”

Section: B Image Segmentationmentioning

confidence: 99%

Reviewing continual learning from the perspective of human-level intelligence

Chang¹,

Li²,

Peng³

et al. 2021

Preprint

View full text Add to dashboard Cite

Humans continual learning (CL) ability is closely related to Stability Versus Plasticity Dilemma that describes how humans achieve ongoing learning capacity and preservation for learned information. The notion of CL has always been present in artificial intelligence (AI) since its births. This paper proposes a comprehensive review on CL. Different from previous reviews that mainly focus on the catastrophic forgetting phenomenon in CL, this paper surveys CL from a more macroscopic perspective based on the Stability Versus Plasticity mechansim. Analogous to biological counterpart, "smart" AI agents are supposed to i) remember previously learned information (information retrospection); ii) infer on new information continuously (information prospection:); iii) transfer useful information (information transfer), to achieve high-level CL. According to the taxonomy, evaluation metrics, algorithms, applications as well as some open issues are then introduced. Our main contributions concern i) recheck CL from the level of artificial general intelligence; ii) provide a detailed and extensive overview on CL topics; iii) present some novel ideas on the potential development of CL.

show abstract

“…only recently been explored and the first experimental studies show that catastrophic forgetting is even more severe than on the classification task [29,31]. Current approaches for class-incremental semantic segmentation re-frame knowledge distillation strategies inspired by previous works on image classification [29,5,22,31]. Although they partially alleviate forgetting, they often fail when multiple incremental steps are performed or when background shift [5] (i.e., change of statistics of the background across learning steps, as it incorporates old or future classes) occurs.…”

Section: Replay Imagesmentioning

confidence: 99%

“…Deep neural networks witnessed remarkable improvements in many fields; however, such models are prone to catastrophic forgetting when they are trained to continuously improve the learned knowledge (e.g., new categories) from progressively provided data [15]. Catastrophic forgetting is a long-standing problem [36,13] which has been recently tackled in a variety of visual tasks such as image classification [21,35,25,43,33], object detection [40,24] and semantic segmentation [29,31,5,22].…”

Section: Related Workmentioning

confidence: 99%

RECALL: Replay-based Continual Learning in Semantic Segmentation

Maracani¹,

Michieli²,

Toldo³

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep networks allow to obtain outstanding results in semantic segmentation, however they need to be trained in a single shot with a large amount of data. Continual learning settings where new classes are learned in incremental steps and previous training data is no longer available are challenging due to the catastrophic forgetting phenomenon. Existing approaches typically fail when several incremental steps are performed or in presence of a distribution shift of the background class. We tackle these issues by recreating no longer available data for the old classes and outlining a content inpainting scheme on the background class. We propose two sources for replay data. The first resorts to a generative adversarial network to sample from the class space of past learning steps. The second relies on web-crawled data to retrieve images containing examples of old classes from online databases. In both scenarios no samples of past steps are stored, thus avoiding privacy concerns. Replay data are then blended with new samples during the incremental steps. Our approach, RECALL, outperforms state-of-the-art methods.* These authors share the first authorship. † Our work was in part supported by the Italian Minister for Education (MIUR) under the "Departments of Excellence" initiative (Law 232/2016).

show abstract

Class-Incremental Learning for Semantic Segmentation Re-Using Neither Old Data Nor Old Labels

Cited by 37 publications

References 21 publications

Continual Coarse-to-Fine Domain Adaptation in Semantic Segmentation

Continual Coarse-to-Fine Domain Adaptation in Semantic Segmentation

Reviewing continual learning from the perspective of human-level intelligence

RECALL: Replay-based Continual Learning in Semantic Segmentation

Contact Info

Product

Resources

About