Attentive History Selection for Conversational Question Answering

Qu, Chen; Liu, Yang; Qiu, Minghui; Zhang, Yongfeng; Chen, Cen; Croft, W. Bruce; Iyyer, Mohit

doi:10.1145/3357384.3357905

Cited by 75 publications

(115 citation statements)

References 29 publications

Supporting

Mentioning

114

Contrasting

Order By: Relevance

“…Given the importance of dealing with answer history, other researchers have proposed to represent answer history using embeddings from pre-trained models (Qu et al 2019). Then, the system includes also a history attention mechanism to help in the selection of items in the history of the dialogue.…”

Section: Technologiesmentioning

confidence: 99%

Survey on evaluation methods for dialogue systems

et al. 2020

View full text Add to dashboard Cite

In this paper, we survey the methods and concepts developed for the evaluation of dialogue systems. Evaluation, in and of itself, is a crucial part during the development process. Often, dialogue systems are evaluated by means of human evaluations and questionnaires. However, this tends to be very cost-and time-intensive. Thus, much work has been put into finding methods which allow a reduction in involvement of human labour. In this survey, we present the main concepts and methods. For this, we differentiate between the various classes of dialogue systems (task-oriented, conversational, and question-answering dialogue systems). We cover each class by introducing the main technologies developed for the dialogue systems and then present the evaluation methods regarding that class.

show abstract

Section: Technologiesmentioning

confidence: 99%

Survey on evaluation methods for dialogue systems

et al. 2020

View full text Add to dashboard Cite

show abstract

“…The MTL methods can be classified into static methods or dynamic methods based on their weighting strategy . In a static method, before training the network, each of the task's weights is set manually, then these weights are fixed throughout the training of the network (Qu et al, 2019b;Yeh and Chen, 2019). In contrast, the dynamic methods initialise each of the task's weights at the beginning of the training and automatically update the weights during the training process (Belharbi et al, 2016;Chen et al, 2018;.…”

Section: Related Workmentioning

confidence: 99%

“…In particular, QuAC introduces a main task, namely Answer Span prediction, which consists in answering a question by extracting text spans from a given passage as well as some auxiliary tasks, namely Yes/No prediction, Follow up prediction and Unanswerable prediction. Recently, Multi-Task Learning (MTL), which is a way to learn multiple different but related tasks simultaneously, has emerged as a popular solution to tackle all these tasks in a uniform model (Qu et al, 2019b). MTL can also be used to leverage the auxiliary tasks to improve the performance of a system on the main task.…”

Section: Introductionmentioning

confidence: 99%

“…MTL can also be used to leverage the auxiliary tasks to improve the performance of a system on the main task. For example, for the QuAC dataset, (Qu et al, 2019b;Yeh and Chen, 2019) adopted an MTL approach that learns the auxiliary tasks and the main task by sharing the encoder, and showed an improvement in the used ConvQA model. MTL methods can be categorised into static or dynamic methods.…”

Section: Introductionmentioning

confidence: 99%

“…Most existing (Qu et al, 2019b;Yeh and Chen, 2019) Conversational Question Answering (Con-vQA) models that leverage Multi-Task Learning (MTL) use the static method with unchanged tasks' weights during the training epochs. For instance, the recently-proposed History Attention Mechanism (HAM) model (Qu et al, 2019b) attempted to apply Multi-Task Learning in order to improve the effectiveness of conversational QA. However, the tasks' weights in the model were unchanged during the training state and emphasise the main task.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multi-Task Learning using Dynamic Task Weighting for Conversational Question Answering

Kongyoung¹,

Macdonald²,

Ounis³

2020

Proceedings of the 5th International Workshop on Search-Oriented Conversational AI (SCAI)

View full text Add to dashboard Cite

Conversational Question Answering (Con-vQA) is a Conversational Search task in a simplified setting, where an answer must be extracted from a given passage. Neural language models, such as BERT, fine-tuned on large-scale ConvQA datasets such as CoQA and QuAC have been used to address this task. Recently, Multi-Task Learning (MTL) has emerged as a particularly interesting approach for developing ConvQA models, where the objective is to enhance the performance of a primary task by sharing the learned structure across several related auxiliary tasks. However, existing ConvQA models that leverage MTL have not investigated the dynamic adjustment of the relative importance of the different tasks during learning, nor the resulting impact on the performance of the learned models. In this paper, we first study the effectiveness and efficiency of dynamic MTL methods including Evolving Weighting, Uncertainty Weighting, and Loss-Balanced Task Weighting, compared to static MTL methods such as the uniform weighting of tasks. Furthermore, we propose a novel hybrid dynamic method combining Abridged Linear for the main task with a Loss-Balanced Task Weighting (LBTW) for the auxiliary tasks, so as to automatically fine-tune task weighting during learning, ensuring that each of the tasks' weights is adjusted by the relative importance of the different tasks. We conduct experiments using QuAC, a large-scale ConvQA dataset. Our results demonstrate the effectiveness of our proposed method, which significantly outperforms both the single-task learning and static task weighting methods with improvements ranging from +2.72% to +3.20% in F1 scores. Finally, our findings show that the performance of using MTL in developing ConvQA model is sensitive to the correct selection of the auxiliary tasks as well as to an adequate balancing of the loss rates of these tasks during training by using LBTW.

show abstract

Combining Impression Feature Representation for Multi-turn Conversational Question Answering

Jing

Hong

Zhao

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Multi-turn conversational Question Answering (ConvQA) is a practical task that requires the understanding of conversation history, such as previous QA pairs, the passage context, and current question. It can be applied to a variety of scenarios with human-machine dialogue. The major challenge of this task is to require the model to consider the relevant conversation history while understanding the passage. Existing methods usually simply prepend the history to the current question, or use the complicated mechanism to model the history. This article proposes an impression feature, which use the word-level inter attention mechanism to learn multi-oriented information from conversation history to the input sequence, including attention from history tokens to each token of the input sequence, and history turn inter attention from different history turns to each token of the input sequence, and self-attention within input sequence, where the input sequence contains a current question and a passage. Then a feature selection method is designed to enhance the useful history turns of conversation and weaken the unnecessary information. Finally, we demonstrate the effectiveness of the proposed method on the QuAC dataset, analyze the impact of different feature selection methods, and verify the validity and reliability of the proposed features through visualization and human evaluation.

show abstract

Attentive History Selection for Conversational Question Answering

Cited by 75 publications

References 29 publications

Survey on evaluation methods for dialogue systems

Survey on evaluation methods for dialogue systems

Multi-Task Learning using Dynamic Task Weighting for Conversational Question Answering

Combining Impression Feature Representation for Multi-turn Conversational Question Answering

Contact Info

Product

Resources

About