Learning to Compose Diversified Prompts for Image Emotion Classification

Deng, Sinuo; Wu, Lifang; Shi, Ge; Xing, Lehao; Jian, Meng; Xiang, Ye; Dong, Ruihai

doi:10.2139/ssrn.4279935

Cited by 6 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Prompt engineering is a task-specific description embedded in inputs to LLMs (e.g., a question format) [15]. It is a critical component affecting zero-shot performance of LLMs on various downstream tasks [1,16,17]. In Section 4.2 we explore various prompts in order to understand the impact of prompt engineering for the entailment task.…”

Section: Prompt Engineeringmentioning

confidence: 99%

“…43.7% A speech seems to express a feeling like {}. [16] 38.0% A transcript seems to express a feeling like {}. [16] 38.9% A conversation seems to express some feelings like {}.…”

Section: Promptsmentioning

confidence: 99%

“…[16] 38.0% A transcript seems to express a feeling like {}. [16] 38.9% A conversation seems to express some feelings like {}. [16] 39.0% The emotion of the conversation is {}.…”

Section: Promptsmentioning

confidence: 99%

“…[16] 38.9% A conversation seems to express some feelings like {}. [16] 39.0% The emotion of the conversation is {}.…”

Section: Promptsmentioning

confidence: 99%

“…To evaluate the prompts, we compute accuracy by comparing the weak labels with the ground-truth. We also examined prompts used in previous emotion recognition studies [16,7] and modified a few visionspecific prompts [16] for our study by replacing words such as "photo" or "image" with "speech". Table 1 shows the accuracy for each prompt.…”

Section: Prompt Engineeringmentioning

confidence: 99%

See 4 more Smart Citations

LanSER: Language-Model Supported Speech Emotion Recognition

Gong¹,

Josh²,

Somandepalli³

et al. 2023

Interspeech 2023

View full text Add to dashboard Cite

Speech emotion recognition (SER) models typically rely on costly human-labeled data for training, making scaling methods to large speech datasets and nuanced emotion taxonomies difficult. We present LanSER, a method that enables the use of unlabeled data by inferring weak emotion labels via pre-trained large language models through weakly-supervised learning. For inferring weak labels constrained to a taxonomy, we use a textual entailment approach that selects an emotion label with the highest entailment score for a speech transcript extracted via automatic speech recognition. Our experimental results show that models pre-trained on large datasets with this weak supervision outperform other baseline models on standard SER datasets when fine-tuned, and show improved label efficiency. Despite being pre-trained on labels derived only from text, we show that the resulting representations appear to model the prosodic content of speech.

show abstract

Section: Prompt Engineeringmentioning

confidence: 99%

“…43.7% A speech seems to express a feeling like {}. [16] 38.0% A transcript seems to express a feeling like {}. [16] 38.9% A conversation seems to express some feelings like {}.…”

Section: Promptsmentioning

confidence: 99%

“…[16] 38.0% A transcript seems to express a feeling like {}. [16] 38.9% A conversation seems to express some feelings like {}. [16] 39.0% The emotion of the conversation is {}.…”

Section: Promptsmentioning

confidence: 99%

“…[16] 38.9% A conversation seems to express some feelings like {}. [16] 39.0% The emotion of the conversation is {}.…”

Section: Promptsmentioning

confidence: 99%