Crowd Score: A Method for the Evaluation of Jokes using Large Language Model AI Voters as Judges

Fabricio, Goes,; Zhou, Zisen; Sawicki, Piotr; Grzes, Marek; Brown, Daniel G.

doi:10.48550/arxiv.2212.11214

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In poetry, LLMs have demonstrated a basic grasp of poetic structure and form, yet they struggle significantly with capturing the essence of metaphorical and allegorical content [9,13]. Another key observation is the issue of cultural contextualization, where models lack the depth to understand historical and literary references integral to Chinese poetry [13,28,29,30,31]. Studies have shown that despite high accuracy in general language tasks, LLMs often produce outputs in poetry tasks that lack emotional depth, cultural resonance and sometimes even explainability [32,33,34,35,36].…”

Section: Llm and Chinese Language Processingmentioning

confidence: 99%

Enhancing Simplified Chinese Poetry Comprehension in LLaMA-7B: A Novel Approach to Mimic Mixture of Experts Effect

Zhang,

Chen

2023

Preprint

View full text Add to dashboard Cite

This study explored the potential of manual augmentation in enhancing the comprehension and translation capabilities of large language models, specifically focusing on the LLaMA-7B model in the context of Chinese poetry. Chinese poetry, with its rich cultural and historical complexities, presents a unique challenge for AI models predominantly trained on modern English datasets. Our research introduced a novel approach by manually augmenting LLaMA-7B, emulating a mixture-of-experts model integration. This method involved integrating specialized linguistic and cultural processing units, which significantly improved the model's ability to interpret the complex tonal patterns, metaphorical richness, and cultural allusions inherent in Chinese poetry. We conducted rigorous evaluations, measuring the augmented model's performance against expert translations and noting a 23% increase in comprehension accuracy and a 37% reduction in semantic hallucinations. Our findings not only demonstrated the efficacy of manual augmentation in bridging the gap between AI capabilities and the demands of classical literary texts, but also opened new avenues for applying similar techniques to other culturally-rich languages. Our study underscored the importance of cultural and contextual awareness in AI language processing, marking a step towards more advanced and culturally sensitive AI models.

show abstract