What and Where: Learn to Plug Adapters via NAS for Multidomain Learning

Zhao, Hanbin; Zeng, Hao; Qin, Xin; Fu, Yongjian; Wang, Hui; Omar, Bourahla; Li, Xi

doi:10.1109/tnnls.2021.3082316

Cited by 7 publications

(2 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The entire backbone architecture keeps domain-agnostic and is shared across domains while the adapters are domain-specific. A recent study [26] has shown that the choice of adapters and the locations they are plugged in depend on the set of domains. It leverages neural architecture search to figure out what adapter to use and where to add adapters for a given set of domains.…”

Section: B Multi-domain Learningmentioning

confidence: 99%

An Alternative Hard-Parameter Sharing Paradigm for Multi-Domain Learning

et al. 2023

View full text Add to dashboard Cite

Hard-parameter sharing in multi-domain learning (MDL) allows domains to share some model parameters in order to reduce storage cost while improving prediction accuracy. One traditional paradigm of the sharing practice borrows an idea from multi-task learning (MTL), which is to share bottom layers of a deep neural network among domains while using separate top layers for each domain. However, it is unclear whether the effectiveness of sharing bottom parameters in MTL can transfer to MDL or not. Therefore in this work, we revisit this common practice via an empirical study on image classification tasks on a diverse set of visual domains and make two surprising observations. (1) Using separate bottom-layer parameters could achieve significantly better performance than the common practice and this phenomenon holds for the different number of domains jointly trained on different backbone architectures with different quantities of domain-specific parameters. (2) A multi-domain model with a small proportion of domain-specific parameters from bottom layers can achieve competitive performance with independent models trained on each domain separately. Our observations suggest that people adopt the new paradigm of using separate bottom-layer parameters as a stronger baseline for model design in MDL.INDEX TERMS Empirical study, hard-parameter sharing, multi-domain learning.

show abstract

Section: B Multi-domain Learningmentioning

confidence: 99%

An Alternative Hard-Parameter Sharing Paradigm for Multi-Domain Learning

et al. 2023

View full text Add to dashboard Cite

show abstract

“…These works can be categorized into three major families: 1) architectural strategies, 2) rehearsal strategies, 3) regularization strategies. Architectural strategies [1,2,25,29,30,51] keep the learned knowledge from previous tasks and acquire new knowledge from the current task by manipulating the network architecture, e.g., parameter masking, network pruning. Rehearsal strategies [17,27,37,42,50] replay old tasks information when learning the new task, and the past knowledge is memorized by storing old tasks' exemplars or old tasks data distribution via generative models.…”

Section: Related Workmentioning

confidence: 99%

RBC: Rectifying the Biased Context in Continual Semantic Segmentation

Zhao¹,

Yang²,

Fu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Recent years have witnessed a great development of Convolutional Neural Networks in semantic segmentation, where all classes of training images are simultaneously available. In practice, new images are usually made available in a consecutive manner, leading to a problem called Continual Semantic Segmentation (CSS). Typically, CSS faces the forgetting problem since previous training images are unavailable, and the semantic shift problem of the background class. Considering the semantic segmentation as a context-dependent pixel-level classification task, we explore CSS from a new perspective of context analysis in this paper. We observe that the context of old-class pixels in the new images is much more biased on new classes than that in the old images, which can sharply aggravate the old-class forgetting and new-class overfitting. To tackle the obstacle, we propose a biased-context-rectified CSS framework with a context-rectified image-duplet learning scheme and a biased-context-insensitive consistency loss. Furthermore, we propose an adaptive re-weighting class-balanced learning strategy for the biased class distribution. Our approach outperforms state-of-the-art methods by a large margin in existing CSS scenarios.

show abstract