Extrinsic Evaluation of Dialog State Tracking and Predictive Metrics for Dialog Policy Optimization

Abstract

During the recent Dialog State Tracking Challenge (DSTC), a fundamental question was raised: "Would better performance in dialog state tracking translate to better performance of the optimized policy by reinforcement learning?" Also, during the challenge system evaluation, another nontrivial question arose: "Which evaluation metric and schedule would best predict improvement in overall dialog performance?" This paper aims to answer these questions by applying an off-policy reinforcement learning method to the output of each challenge system. The results give a positive answer to the first question. Thus the effort to separately improve the performance of dialog state tracking as carried out in the DSTC may be justified. The answer to the second question also draws several insightful conclusions on the characteristics of different evaluation metrics and schedules.

The Dialog State Tracking Challenge Series: A Review

“…Second, in DSTC2, the question of what to measure was posed differently, as "Which evaluation metric and schedule would best predict improvement in overall dialog performance?" (Lee, 2014). The author uses the data to optimize a reinforcement learning-based dialog manager, then runs a regression analysis to see which metrics are the best predictors of end-to-end dialog performance.…”

“…This has had unforeseen benefits: first, the DSTC data now forms a sort of benchmark for the field, with groups continuing to report results on it after the challenge proper (Lee, 2013;Ma and Fosler-Lussier, 2014b;Zilka and Jurčíček, 2015;Fix and Frezza-Buet, 2015). In addition, the DSTC1-3 corpora have been used to examine which state tracking evaluation metrics correlate with dialog success (Lee, 2014), perform detailed error analyses of state trackers (Smith, 2014), and for dialog act classification and SLU experimentation (Ma and Fosler-Lussier, 2014a;Ferreira et al, 2015). We encourage future challenges to continue this tradition.…”

Dialog History Construction with Long-Short Term Memory for Robust Generative Dialog State Tracking

“…In this paper, we use L2 metric as the loss function since it is found to be most influential to dialog system performance (Lee, 2014). The model is hence optimized to minimize the L2 loss function…”

Optimizing human-interpretable dialog management policy using genetic algorithm

“…A held-out dialog corpus is used as testing set, and the estimated cumulative reward for the testing dialogs when following the target DM policy is used as metric for performance. A similar approach has been taken in evaluating the effect of different dialog state tracker on end-to-end performance of a DM [15]. The estimation of Q-function is similar to Algorithm 2.…”