On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark

Sun, Hao; Xu, Guangxuan; Deng, Jiawen; Cheng, Jiale; Zheng, Chujie; Zhou, Hao; Peng, Nanyun; Zhu, Xiaoyan; Huang, Minlie

doi:10.18653/v1/2022.findings-acl.308

Cited by 20 publications

(25 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Examples of safety issues exposed in dialogue system responses. These cases are collected from existing datasets [91,139,170]. The last column gives explanations of why the responses are unsafe.…”

Section: Usermentioning

confidence: 99%

“…Sentence-level toxic content is usually expressed with explicit toxicity and is usually context-independent, such as "Guys without jobs are all idiots.". In a dialogue, however, each sentence is uttered based on historical utterances and contains more complex and implicit information, thus, its toxicity could be amplified or even vanish due to the nature of interaction and context-sensitiveness [139].…”

Section: Usermentioning

confidence: 99%

“…For example, recent works emphasize the physical and mental harm of inappropriate advice by dialogue system [32,69,139], which is hardly ever mentioned in previous works. To view safety as a whole and help structure it, several works try to propose a unified taxonomy to classify safety issues from top to bottom [32,139,151,165]. However, different research institutions have respective stances and opinions on safety classification and the concept "safety" itself is ill-defined till now.…”

Section: Introductionmentioning

confidence: 99%

“…As the very first task, automatic hate speech detection has been applied since many years ago [123,136]. As the development of language and culture, along with the intelligence improvement of dialogue systems, the requirements towards dialogue systems naturally step to a whole new high level and the scope of safety gradually becomes larger [115,139]. For example, recent works emphasize the physical and mental harm of inappropriate advice by dialogue system [32,69,139], which is hardly ever mentioned in previous works.…”

Section: Introductionmentioning

confidence: 99%

“…As the development of language and culture, along with the intelligence improvement of dialogue systems, the requirements towards dialogue systems naturally step to a whole new high level and the scope of safety gradually becomes larger [115,139]. For example, recent works emphasize the physical and mental harm of inappropriate advice by dialogue system [32,69,139], which is hardly ever mentioned in previous works. To view safety as a whole and help structure it, several works try to propose a unified taxonomy to classify safety issues from top to bottom [32,139,151,165].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Recent Advances towards Safe, Responsible, and Moral Dialogue Systems: A Survey

Deng¹,

Sun²,

Zhang³

et al. 2023

Preprint

View full text Add to dashboard Cite

With the development of artificial intelligence, dialogue systems have been endowed with amazing chitchat capabilities, and there is widespread interest and discussion about whether the generated contents are socially beneficial. In this paper, we present a new perspective of research scope towards building a safe, responsible, and modal dialogue system, including 1) abusive and toxic contents, 2) unfairness and discrimination, 3) ethics and morality issues, and 4) risk of misleading and privacy information. Besides, we review the mainstream methods for evaluating the safety of large models from the perspectives of exposure and detection of safety issues. The recent advances in methodologies for the safety improvement of both end-to-end dialogue systems and pipeline-based models are further introduced. Finally, we discussed six existing challenges towards responsible AI: explainable safety monitoring, continuous learning of safety issues, robustness against malicious attacks, multimodal information processing, unified research framework, and multidisciplinary theory integration. We hope this survey will inspire further research toward safer dialogue systems. 1

show abstract

Section: Usermentioning

confidence: 99%

Section: Usermentioning

confidence: 99%