Learning to Compose Diversified Prompts for Image Emotion Classification

Deng, Sinuo; Wu, Lifang; Shi, Ge; Xing, Lehao; Meng, Jian

doi:10.48550/arxiv.2201.10963

Cited by 2 publications

(2 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yi et al (2022) developed a contextual information and commonsense-based prompt learning model for conversational sentiment analysis, demonstrating superior performance over state-of-the-art models. Deng et al (2022b) also introduced a prompt tuning method that mimicked the pre-training objective of contrastive language-image pre-training (CLIP). It thus could leverage the rich image and text semantics for image emotion classification.…”

Section: Prompt Learningmentioning

confidence: 99%

Multi-Modal Attentive Prompt Learning for Few-shot Emotion Recognition in Conversations

Liang,

Tu,

et al. 2024

jair

View full text Add to dashboard Cite

Emotion recognition in conversations (ERC) has emerged as an important research area in Natural Language Processing and Affective Computing, focusing on accurately identifying emotions within the conversational utterance. Conventional approaches typically rely on labeled training samples for fine-tuning pre-trained language models (PLMs) to enhance classification performance. However, the limited availability of labeled data in real-world scenarios poses a significant challenge, potentially resulting in diminished model performance. In response to this challenge, we present the Multi-modal Attentive Prompt (MAP) learning framework, tailored specifically for few-shot emotion recognition in conversations. The MAP framework consists of four integral modules: multi-modal feature extraction for the sequential embedding of text, visual, and acoustic inputs; a multi-modal prompt generation module that creates six manually-designed multi-modal prompts; an attention mechanism for prompt aggregation; and an emotion inference module for emotion prediction. To evaluate our proposed model’s efficacy, we conducted extensive experiments on two widely recognized benchmark datasets, MELD and IEMOCAP. Our results demonstrate that the MAP framework outperforms state-of-the-art ERC models, yielding notable improvements of 3.5% and 0.4% in micro F1 scores. These findings highlight the MAP learning framework’s ability to effectively address the challenge of limited labeled data in emotion recognition, offering a promising strategy for improving ERC model performance.

show abstract

Section: Prompt Learningmentioning

confidence: 99%

Multi-Modal Attentive Prompt Learning for Few-shot Emotion Recognition in Conversations

Liang,

Tu,

et al. 2024

jair

View full text Add to dashboard Cite

show abstract

“…To overcome the limitations of fixed prompts, many recent learned prompt works [19,60] propose to incorporate dataset-specific context information by using learnable prompt context vectors that can be catered to work best for each particular class. These learned prompts are biased towards seen classes.…”

Section: Prompt Learningmentioning

confidence: 99%

A Language-Guided Benchmark for Weakly Supervised Open Vocabulary Semantic Segmentation

Pandey¹,

Chasmai²,

Natarajan³

et al. 2023

Preprint

View full text Add to dashboard Cite

First method to explore multiple and related Open Vocabulary Semantic Segmentation inductive tasks in a weakly supervised setting without using external datasets and fine-tuning• First method to handle weakly supervised generalized zero-shot segmentation, zero-shot segmentation and few-shot segmentation with a single training procedure using a frozen vision-language model• Propose a novel and scalable mean instance aware prompt learning that generates highly generalizable prompts, handles domain shift across the datasets and generalizes efficiently to unseen classes• The flexible design allows easy modification and optimization of different components as and when required• The proposed method beats existing weakly supervised baselines by large margins while being competitive with pixel-based methods

show abstract

Learning to Compose Diversified Prompts for Image Emotion Classification

Cited by 2 publications

References 4 publications

Multi-Modal Attentive Prompt Learning for Few-shot Emotion Recognition in Conversations

Multi-Modal Attentive Prompt Learning for Few-shot Emotion Recognition in Conversations

A Language-Guided Benchmark for Weakly Supervised Open Vocabulary Semantic Segmentation

Contact Info

Product

Resources

About