Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model

Nico, Daheim,; Thulke, David; Dugast, Christian; Ney, Hermann

doi:10.18653/v1/2022.findings-emnlp.98

Cited by 2 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Mitigating hallucinations in information-seeking dialogue has achieved increased interest recently with the omnipresence of large language models (Wang et al, 2023b;Chuang et al, 2023;Daheim et al, 2022Daheim et al, , 2023. Previously proposed methods can be largely divided into those which increase factuality of pretrained models via further training or modification of the generation procedure.…”

Section: Related Workmentioning

confidence: 99%

Data Augmentation and Learned Layer Aggregation for Improved Multilingual Language Understanding in Dialogue

Razumovskaia¹,

Vulić²,

Korhonen³

2022

Findings of the Association for Computational Linguistics: ACL 2022

View full text Add to dashboard Cite

Scaling dialogue systems to a multitude of domains, tasks and languages relies on costly and time-consuming data annotation for different domain-task-language configurations. The annotation efforts might be substantially reduced by the methods that generalise well in zero-and few-shot scenarios, and also effectively leverage external unannotated data sources (e.g., Web-scale corpora). We propose two methods to this aim, offering improved dialogue natural language understanding (NLU) across multiple languages: 1) Multi-SentAugment, and 2) LayerAgg. Multi-SentAugment is a self-training method which augments available (typically few-shot) training data with similar (automatically labelled) in-domain sentences from large monolingual Web-scale corpora. LayerAgg learns to select and combine useful semantic information scattered across different layers of a Transformer model (e.g., mBERT); it is especially suited for zero-shot scenarios as semantically richer representations should strengthen the model's cross-lingual capabilities. Applying the two methods with state-of-the-art NLU models obtains consistent improvements across two standard multilingual NLU datasets covering 16 diverse languages. The gains are observed in zero-shot, few-shot, and even in full-data scenarios. The results also suggest that the two methods achieve a synergistic effect: the best overall performance in few-shot setups is attained when the methods are used together.

show abstract

Section: Related Workmentioning

confidence: 99%

Data Augmentation and Learned Layer Aggregation for Improved Multilingual Language Understanding in Dialogue

Razumovskaia¹,

Vulić²,

Korhonen³

2022

Findings of the Association for Computational Linguistics: ACL 2022

View full text Add to dashboard Cite

show abstract

Task-Oriented Document-Grounded Dialog Systems by HLTPR@RWTH for DSTC9 and DSTC10

Thulke

Nico

Dugast

et al. 2024

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

This paper summarizes our contributions to the document-grounded dialog tasks at the 9th and 10th Dialog System Technology Challenges (DSTC9 and DSTC10). In both iterations the task consists of three subtasks: first detect whether the current turn is knowledge seeking, second select a relevant knowledge document, and third generate a response grounded on the selected document. For DSTC9 we proposed different approaches to make the selection task more efficient. The best method, Hierarchical Selection, actually improves the results compared to the original baseline and gives a speedup of 24x. In the DSTC10 iteration of the task, the challenge was to adapt systems trained on written dialogs to perform well on noisy automatic speech recognition transcripts. Therefore, we proposed data augmentation techniques to increase the robustness of the models as well as methods to adapt the style of generated responses to fit well into the proceeding dialog. Additionally, we proposed a noisy channel model that allows for increasing the factuality of the generated responses. In addition to summarizing our previous contributions, in this work, we also report on a few small improvements and reconsider the automatic evaluation metrics for the generation task which have shown a low correlation to human judgments.

show abstract

Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model

Cited by 2 publications

References 0 publications

Data Augmentation and Learned Layer Aggregation for Improved Multilingual Language Understanding in Dialogue

Data Augmentation and Learned Layer Aggregation for Improved Multilingual Language Understanding in Dialogue

Task-Oriented Document-Grounded Dialog Systems by HLTPR@RWTH for DSTC9 and DSTC10

Contact Info

Product

Resources

About