Optimization of Information-Seeking Dialogue Strategy for Argumentation-Based Dialogue System

Abstract

Argumentation-based dialogue systems, which can handle and exchange arguments through dialogue, have been widely researched. It is required that these systems have sufficient supporting information to argue their claims rationally; however, the systems often do not have enough of such information in realistic situations. One way to fill in the gap is acquiring such missing information from dialogue partners (information-seeking dialogue). Existing informationseeking dialogue systems are based on handcrafted dialogue strategies that exhaustively examine missing information. However, the proposed strategies are not specialized in collecting information for constructing rational arguments. Moreover, the number of system's inquiry candidates grows in accordance with the size of the argument set that the system deal with. In this paper, we formalize the process of information-seeking dialogue as Markov decision processes (MDPs) and apply deep reinforcement learning (DRL) for automatically optimizing a dialogue strategy. By utilizing DRL, our dialogue strategy can successfully minimize objective functions, the number of turns it takes for our system to collect necessary information in a dialogue. We conducted dialogue experiments using two datasets from different domains of argumentative dialogue. Experimental results show that the proposed formalization based on MDPs works well, and the policy optimized by DRL outperformed existing heuristic dialogue strategies.

A study of interactive robot architecture through the practical implementation of conversational android

“…The user’s experience can be acquired by 1H5W questions, but if the system repeats asking the user’s experience by 1H5W questions, it becomes annoying for the user. In information-seeking dialogues, optimizing the number of questions has been tried by using an inference system (e.g., Katsumi et al, 2019 ). The inference-based system is much more necessary for a daily dialogue to maintain a comfortable dialogue.…”

Machine Learning for Utility Prediction in Argument-Based Computational Persuasion

“…In order for an APS to choose the moves it makes in the dialogue (i.e., the arguments it presents), it needs to use a strategy. Within the literature on computational models of arguments, key approaches to strategies are: Mechanism design where it is a one-step process rather for multistep dialogues (Rahwan and Larson 2008;Rahwan, Larson, and Tohmé 2009;Fan and Toni 2012); Planning systems to optimize choice of arguments based on belief in premises (Black, Coles, and Bernardini 2014;Black, Coles, and Hampson 2017) or minimize the number of moves made (Atkinson, Bench-Capon, and Bench-Capon 2012); Machine learning of strategies such as Reinforcement Learning (Monteserin and Amandi 2013;Rosenfeld and Kraus 2016b;Rach, Minker, and Ultes 2018;Katsumi et al 2018;Riveret et al 2019;Alahmari 2020) and transfer learning (Rosenfeld and Kraus 2016a); Probabilistic methods to select a move based on what an agent believes the other is aware of (Rienstra, Thimm, and Oren 2013), to approximately predict the argument an opponent might put forward based on data about the moves made by the opponent in previous dialogues (Hadjinikolis et al 2013), using a decisiontheoretic lottery (Hunter and Thimm 2016), using POMDPs when there is uncertainty about the internal state of the opponent (Hadoux et al 2015); and Local strategies based for example on the concerns (i.e., issues raised or addressed by an argument) of the persuadee (Hadoux and Hunter 2019;Chalaguine and Hunter 2020).…”