SIAST: A Slot Imbalance-Aware Self-Training Scheme for Semi-Supervised Slot Filling

Liu, Jiachi; Xiong, Shiyong; He, Yuehuan; Zhou, Tong; Wang, Liwen; Li, Xuefeng; Xiao, Bo

doi:10.1109/icassp49357.2023.10096302

Cited by 2 publications

(1 citation statement)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…evaluate the robustness of ChatGPT from the out-of-distribution (OOD) perspective. A series works by Liu et al (2023); ; Dong et al (2022a explore the impact of input perturbation problems on model performance Next, I will first give you a set of sentences , which will be recorded as Set 1. First,Please classify the sentences in Set 1 into 5 (Number of Clusters) categories according to their intentions.…”

Section: Large Language Modelsmentioning

confidence: 99%

Large Language Models Meet Open-World Intent Discovery and Recognition: An Evaluation of ChatGPT

Song,

He,

Wang

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

The tasks of out-of-domain (OOD) intent discovery and generalized intent discovery (GID) aim to extend a closed intent classifier to openworld intent sets, which is crucial to taskoriented dialogue (TOD) systems. Previous methods address them by fine-tuning discriminative models. Recently, although some studies have been exploring the application of large language models (LLMs) represented by ChatGPT to various downstream tasks, it is still unclear for the ability of ChatGPT to discover and incrementally extent OOD intents. In this paper, we comprehensively evaluate ChatGPT on OOD intent discovery and GID, and then outline the strengths and weaknesses of ChatGPT. Overall, ChatGPT exhibits consistent advantages under zero-shot settings, but is still at a disadvantage compared to fine-tuned models. More deeply, through a series of analytical experiments, we summarize and discuss the challenges faced by LLMs including clustering, domain-specific understanding, and cross-domain in-context learning scenarios. Finally, we provide empirical guidance for future directions to address these challenges. 1

show abstract

Section: Large Language Modelsmentioning

confidence: 99%