The Dialog State Tracking Challenge Series: A Review

Williams, Jason D.; Raux, Antoine; Henderson, Matthew

doi:10.5087/dad.2016.301

Cited by 167 publications

(124 citation statements)

References 53 publications

Supporting

Mentioning

124

Contrasting

Order By: Relevance

“…The methods of dialogue state monitoring can be divided into a rules-based approach, statistics and a deep learning system. The use rule-based heuristics and calculate the confidence scores of the N-best candidates produced to determine the correct dialogue states from the performance of the natural language comprehension module [4][5][6][7]. The purpose is monitor the details needed to track the status of the dialogue.…”

Section: Related Workmentioning

confidence: 99%

Dialogue state tracking accuracy improvement by distinguishing slot-value pairs and dialogue behaviour

Alhussayni

Zamyatin

Al–Shamery

2021

IJEECS

View full text Add to dashboard Cite

<div><p>Dialog state tracking (DST) plays a critical role in cycle life of a task-oriented dialogue system. DST represents the goals of the consumer at each step by dialogue and describes such objectives as a conceptual structure comprising slot-value pairs and dialogue actions that specifically improve the performance and effectiveness of dialogue systems. DST faces several challenges: diversity of linguistics, dynamic social context and the dissemination of the state of dialogue over candidate values both in slot values and in dialogue acts determined in ontology. In many turns during the dialogue, users indirectly refer to the previous utterances, and that produce a challenge to distinguishing and use of related dialogue history, Recent methods used and popular for that are ineffective. In this paper, we propose a dialogue historical context self-Attention framework for DST that recognizes relevant historical context by including previous user utterance beside current user utterances and previous system actions where specific slot-value piers variations and uses that together with weighted system utterance to outperform existing models by recognizing the related context and the relevance of a system utterance. For the evaluation of the proposed model the WoZ dataset was used. The implementation was attempted with the prior user utterance as a dialogue encoder and second by the additional score combined with all the candidate slot-value pairs in the context of previous user utterances and current utterances. The proposed model obtained 0.8 per cent better results than all state-of-the-art methods in the combined precision of the target, but this is not the turnaround challenge for the submission.</p></div>

show abstract

Section: Related Workmentioning

confidence: 99%

Dialogue state tracking accuracy improvement by distinguishing slot-value pairs and dialogue behaviour

Alhussayni

Zamyatin

Al–Shamery

2021

IJEECS

View full text Add to dashboard Cite

show abstract

“…These multi-domain dialogues (Example goals user try to achieve in Appendix Table 10) are representative of end user interactions with Alexa and were randomly sampled from two dialogue systems. Dialogue-system A uses a pipelined modular dialogue agent comprising of ASR, NLU, State Tracker, Dialogue Policy and Natural Language Generation components (Williams et al, 2016). Dialogue-system B is an end-to-end neural model (Ritter et al, 2011;Shah et al, 2018) that shares only the ASR component with system A (Fig.…”

Section: Dialogue Quality Datamentioning

confidence: 99%

Joint Turn and Dialogue level User Satisfaction Estimation on Multi-Domain Conversations

Bodigutla¹,

Tiwari²,

Matsoukas³

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Dialogue level quality estimation is vital for optimizing data driven dialogue management. Current automated methods to estimate turn and dialogue level user satisfaction employ hand-crafted features and rely on complex annotation schemes, which reduce the generalizability of the trained models. We propose a novel user satisfaction estimation approach which minimizes an adaptive multi-task loss function in order to jointly predict turn-level Response Quality labels provided by experts and explicit dialogue-level ratings provided by end users. The proposed BiLSTM based deep neural net model automatically weighs each turn's contribution towards the estimated dialogue-level rating, implicitly encodes temporal dependencies, and removes the need to hand-craft features.On dialogues sampled from 28 Alexa domains, two dialogue systems and three user groups, the joint dialogue-level satisfaction estimation model achieved up to an absolute 27% (0.43 → 0.70) and 7% (0.63 → 0.70) improvement in linear correlation performance over baseline deep neural net and benchmark Gradient boosting regression models, respectively.

show abstract

“…Frame-based dialog systems keep track of the dialog state by tracking a set of slots and values, representing the user's goals [36]. Many approaches to state tracking, including rule-based systems, graphical models, and artificial neural networks, have been evaluated on manually labeled benchmark datasets [37], [38].…”

Section: E Dialog Systemsmentioning

confidence: 99%

“…For prediction, the customer's utterance vectorizations were input and the typical utterance of the predicted speech cluster was the final output. RNNs are a widely used machine learning approach for end-to-end dialog systems [35], [51] and other natural language processing tasks [52], [53]; therefore, they are a suitable for comparison.…”

Section: A Experimental Designmentioning

confidence: 99%

Modeling Interaction Structure for Robot Imitation Learning of Human Social Behavior

Doering

Glas

Ishiguro

2019

IEEE Trans. Human-Mach. Syst.

View full text Add to dashboard Cite

This study presents a learning-by-imitation technique that learns social robot interaction behaviors from natural humanhuman interaction data and requires minimum input from a designer. To solve the problem of responding to ambiguous human actions, a novel topic clustering algorithm based on action cooccurrence frequencies is introduced. The system learns humanreadable rules that dictate which action the robot should take, based on the most recent human action and the current estimated topic of conversation. The technique is demonstrated in a scenario where the robot learns to play the role of a travel agent. The proposed technique outperformed several baseline techniques in qualitative and quantitative evaluations. It responded more accurately to ambiguous questions and participants found it was easier to understand, provided more information, and required less effort to interact with.

show abstract

The Dialog State Tracking Challenge Series: A Review

Cited by 167 publications

References 53 publications

Dialogue state tracking accuracy improvement by distinguishing slot-value pairs and dialogue behaviour

Dialogue state tracking accuracy improvement by distinguishing slot-value pairs and dialogue behaviour

Joint Turn and Dialogue level User Satisfaction Estimation on Multi-Domain Conversations

Modeling Interaction Structure for Robot Imitation Learning of Human Social Behavior

Contact Info

Product

Resources

About