Exploiting multi-scale contextual prompt learning for zero-shot semantic segmentation

Wang, Yiqi; Tian, Yingjie

doi:10.1016/j.displa.2023.102616

Cited by 2 publications

(2 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where Equation (7) represents the mapping of semantic features to visual space, v represents attribute semantic features, f represents sample visual features, and 1 W is a learnable mapping matrix to compute the correlation between visual features and attribute semantic features. Similarly, Equation ( 8) represents the mapping of visual features to attribute semantic space, and 2 W is a learnable mapping matrix to compute the correlation between attribute semantic features and visual features.…”

Section: Feature Mapping Modulementioning

confidence: 99%

“…The difference between zero-shot image classification and traditional image classification is whether the training set contains the category samples from the test set, as shown in Figure 1. Since its development, the idea of zero-shot learning has attracted much attention and has been introduced into various tasks, such as zero-shot image retrieval [2,3] and zero-shot video classification [4] tasks in the field of computer vision, zero-shot sketch retrieval [5] tasks in the cross-modal field, zero-shot semantic segmentation [6,7] and zeroshot text classification [8] tasks in the field of natural language processing, and zero-shot visual quizzing [9] tasks in the multimodal field. The research object of this paper is the zero-shot image classification task, i.e., the zero-shot learning idea is applied to the task of image classification.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Embedded Zero-Shot Image Classification Based on Bidirectional Feature Mapping

Sun,

Zhen,

Liu

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

The zero-shot image classification technique aims to explore the semantic information shared between seen and unseen classes through visual features and auxiliary information and, based on this semantic information, to complete the knowledge migration from seen to unseen classes in order to complete the classification of unseen class images. Previous zero-shot work has either not extracted enough features to express the relationship between the sample classes or has only used a single feature mapping method, which cannot fully explore the information contained in the features and the connection between the visual–semantic features. To address the above problems, this paper proposes an embedded zero-shot image classification model based on bidirectional feature mapping (BFM). It mainly contains a feature space mapping module, which is dominated by a bidirectional feature mapping network and supplemented with a mapping network from visual to category label semantic feature space. Attention mechanisms based on attribute guidance and visual guidance are further introduced to weight the features to reduce the difference between visual and semantic features to alleviate the modal difference problem, and then the category calibration loss is utilized to assign a larger weight to the unseen class to alleviate the seen class bias problem. The BFM model proposed in this paper has been experimented on three public datasets CUB, SUN, and AWA2, and has achieved 71.9%, 62.8%, and 69.3% and 61.6%, 33.2%, and 66.6% accuracies under traditional and generalized zero-sample image classification settings, respectively. The experimental results verify the superiority of the BFM model in the field of zero-shot image classification.

show abstract

Section: Feature Mapping Modulementioning

confidence: 99%