Enhancing Visual Dialog Questioner with Entity-based Strategy Learning and Augmented Guesser

Abstract

Considering the importance of building a good Visual Dialog (VD) Questioner, many researchers study the topic under a Q-Bot-A-Bot image-guessing game setting, where the Questioner needs to raise a series of questions to collect information of an undisclosed image. Despite progress has been made in Supervised Learning (SL) and Reinforcement Learning (RL), issues still exist. Firstly, previous methods do not provide explicit and effective guidance for Questioner to generate visually related and informative questions. Secondly, the effect of RL is hampered by an incompetent component, i.e., the Guesser, who makes image predictions based on the generated dialogs and assigns rewards accordingly. To enhance VD Questioner: 1) we propose a Related entity enhanced Questioner (ReeQ) that generates questions under the guidance of related entities and learns entity-based questioning strategy from human dialogs; 2) we propose an Augmented Guesser (AugG) that is strong and is optimized for the VD setting especially. Experimental results on the VisDial v1.0 dataset show that our approach achieves state-of-theart performance on both image-guessing task and question diversity. Human study further proves that our model generates more visually related, informative and coherent questions.

Improving the Robustness of Knowledge-Grounded Dialogue via Contrastive Learning

“…Knowledge-Grounded Dialogue (KGD) aims to generate an informative response based on a given dialogue context and external knowledge to improve the usefulness and meaningfulness of the generated responses (Ghazvininejad et al 2017;Zhou et al 2018;Liu et al 2019;Kim, Ahn, and Kim 2020;Li et al 2021;Rashkin et al 2021;Wu et al 2022;Sun et al 2023). As for the choice of knowledge source, structural knowledge graphs (KGs) are proven options (Moon et al 2019;Jung, Son, and Lyu 2020;Wu et al 2022), which consist of a lot of knowledge facts that are frequently used in daily life (Zheng et al 2021;Zhang et al 2022;Cao et al Have you ever heard about Leonardo? Oh, absolutely!…”