Cloud-Device Collaborative Adaptation to Continual Changing Environments in the Real-world

Gan, Yulu; Pan, Mingjie; Zhang, Rongyu; Ling, Zijian; Zhao, Lingran; Liu, Jiaming; Zhang, Shanghang

doi:10.48550/arxiv.2212.00972

Cited by 2 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The reformulated image x will serve as the input for our model instead. (Gal and Ghahramani 2016;Guan et al 2021;Roy et al 2022;Gan et al 2022b) and tactfully place trainable parameters of SVDP on the pixel with large distribution shifts.…”

Section: Methods Preliminariesmentioning

confidence: 99%

See 1 more Smart Citation

Exploring Sparse Visual Prompt for Domain Adaptive Dense Prediction

Yang,

Wu,

Liu

et al. 2024

AAAI

View full text Add to dashboard Cite

The visual prompts have provided an efficient manner in addressing visual cross-domain problems. Previous works introduce domain prompts to tackle the classification Test-Time Adaptation (TTA) problem by placing image-level prompts on the input and fine-tuning prompts for each target domain. However, since the image-level prompts mask out continuous spatial details in the prompt-allocated region, it will suffer from inaccurate contextual information and limited domain knowledge extraction, particularly when dealing with dense prediction TTA problems. To overcome these challenges, we propose a novel Sparse Visual Domain Prompts (SVDP) approach, which applies minimal trainable parameters (e.g., 0.1%) to pixels across the entire image and reserves more spatial information of the input. To better apply SVDP in extracting domain-specific knowledge, we introduce the Domain Prompt Placement (DPP) method to adaptively allocates trainable parameters of SVDP on the pixels with large distribution shifts. Furthermore, recognizing that each target domain sample exhibits a unique domain shift, we design Domain Prompt Updating (DPU) strategy to optimize prompt parameters differently for each sample, facilitating efficient adaptation to the target domain. Extensive experiments were conducted on widely-used TTA and continual TTA benchmarks, and our proposed method achieves state-of-the-art performance in both semantic segmentation and depth estimation tasks.

show abstract

Section: Methods Preliminariesmentioning

confidence: 99%

“…Meanwhile, we can also obtain m sets of probabilities through the simpler method of input image resolution augmentation. Inspired by (Roy et al 2022;Gan et al 2022b), we calculate the uncertainty value (Eq. ( 2)) of the input and figure out the pixel-wise degree of domain shift.…”

Section: Domain Prompt Placingmentioning

confidence: 99%

Exploring Sparse Visual Prompt for Domain Adaptive Dense Prediction

Yang,

Wu,

Liu

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…By locally processing real-time data, this approach eliminates the need for data transfer to cloud devices, saving significant processing time, data transmission, and resource consumption. This technique has matured in the field of autonomous driving [2,3]. Subsequently, the faster and more abundant acquisition of remote sensing data has created new standards and requirements for deep learning models, making it essential for these models to adapt and learn continuously over time.…”

Section: Introductionmentioning

confidence: 99%

DILRS: Domain-Incremental Learning for Semantic Segmentation in Multi-Source Remote Sensing Data

Xue

Cao

et al. 2023

Remote Sensing

View full text Add to dashboard Cite

With the exponential growth in the speed and volume of remote sensing data, deep learning models are expected to adapt and continually learn over time. Unfortunately, the domain shift between multi-source remote sensing data from various sensors and regions poses a significant challenge. Segmentation models face difficulty in adapting to incremental domains due to catastrophic forgetting, which can be addressed via incremental learning methods. However, current incremental learning methods mainly focus on class-incremental learning, wherein classes belong to the same remote sensing domain, and neglect investigations into incremental domains in remote sensing. To solve this problem, we propose a domain-incremental learning method for semantic segmentation in multi-source remote sensing data. Specifically, our model aims to incrementally learn a new domain while preserving its performance on previous domains without accessing previous domain data. To achieve this, our model has a unique parameter learning structure that reparametrizes domain-agnostic and domain-specific parameters. We use different optimization strategies to adapt to domain shift in incremental domain learning. Additionally, we adopt multi-level knowledge distillation loss to mitigate the impact of label space shift among domains. The experiments demonstrate that our method achieves excellent performance in domain-incremental settings, outperforming existing methods with only a few parameters.

show abstract

Cloud-Device Collaborative Adaptation to Continual Changing Environments in the Real-world

Cited by 2 publications

References 0 publications

Exploring Sparse Visual Prompt for Domain Adaptive Dense Prediction

Exploring Sparse Visual Prompt for Domain Adaptive Dense Prediction

DILRS: Domain-Incremental Learning for Semantic Segmentation in Multi-Source Remote Sensing Data

Contact Info

Product

Resources

About