Follow-up question handling in the IMIX and Ritel systems: A comparative study

Abstract

One of the basic topics of QA dialogue systems is how follow-up questions should be interpreted by a QA system. In this paper, we shall discuss our experience with the IMIX and Ritel systems, for both of which a follow-up question handling scheme has been developed, and corpora have been collected. These two systems are each other's opposites in many respects: IMIX is multimodal, non-factoid, black-box QA, while Ritel is speech, factoid, keyword-based QA. Nevertheless, we will show that they are quite comparable, and that it is fruitful to examine the similarities and differences. We shall look at how the systems are composed, and how real, non-expert, users interact with the systems. We shall also provide comparisons with systems from the literature where possible, and indicate where open issues lie and in what areas existing systems may be improved. We conclude that most systems have a common architecture with a set of common subtasks, in particular detecting follow-up questions and finding referents for them. We characterise these tasks using the typical techniques used for performing them, and data from our corpora. We also identify a special type of follow-up question, the discourse question, which is asked when the user is trying to understand an answer, and propose some basic methods for handling it.

Interactive Question Answering

“…Interactive question answering systems give users the opportunity to ask questions and refine them on the basis of feedback received from the system until they find the answer needed (Webb and Webber, 2009). Some interactive question answering systems also allow follow-up questions when the user wants to obtain more information related to the question or clarify the answer, so that IQA becomes an iterative process (Van Schooten et al, 2009). For this reason, we can consider interactive question answering systems a step forward from simple question answering systems by offering users a more natural way of finding the information they need.…”

Answering contextual questions based on ontologies and question templates

“…These types of follow-up question might require different processing strategies. Hence, important sub-tasks within the IQA community are the identification of a typology of questions, detection of the identified types, and the development of different context fusion strategies for each type [17]. In general, the currently employed approaches for context fusion are mainly classified into two types.…”

Analyzing Interactive QA Dialogues Using Logistic Regression Models

“…The importance of evaluating IQA against real user questions and the need to consider preceding system answers has already been emphasized [2][3][4]1]. In these earlier studies, dialogue corpora were collected via Wizard of Oz experiments and/or by giving specific tasks to the users.…”

“…Fine grained typologies of questions have been suggested [2,3,5], and different processing strategies have been proposed for the identified types. We consider the basic distinction between topic shift and topic continuation, and we propose a generalized linear model framework in which this distinction is automatically detected and used to improve the answering performance.…”

“…These types of FU Qs might require different processing strategies. Hence, important sub-tasks within the IQA community are the identification of a typology of questions, automatic detection of the identified types, and the development of different context fusion strategies for each type [2,3,1].…”