Learning concepts through conversations in spoken dialogue systems

Jia, Robin; Heck, Larry; Hakkani‐Tür, Dilek; Nikolov, Georgi

doi:10.1109/icassp.2017.7953253

Cited by 6 publications

(14 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also did an error analysis and found that nearly half of the span prediction errors are due to a mismatch on preposition (e.g. "on Sept 15" vs "Sept 15"), which is not unexpected since span annotations in [8] are not consistent on the inclusion of prepositions.…”

Section: Definition Understanding Resultsmentioning

confidence: 82%

“…Datasets: We evaluate our results on two datasets: 1) The dataset published by [8] which consists of both personal concepts and generic slot values for five slot types namely date, time, location, people and restaurant-name. While the dataset in [8] pertains to the problem that we have attempted to solve in this paper, we find that it does not include challenging real-world scenarios such as the ones described in Section 3 which are addressed by our models. We hence used a second internal dataset collected from crowd-sourcing which addressed the challenges of realistic multi-turn teaching sessions.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…Public Datasets: Table 3 shows the performance of our definition understanding model on personalized test set from [8] with three Transformer-based encoders. From the results, we observe that model with RoBERTa [16] encoder achieves the best performance in terms of precision, recall and F1-score.…”

Section: Definition Understanding Resultsmentioning

confidence: 99%

“…The problem of predicting segments of an utterance which a base NLU system could not interpret, was tackled by Jia et al [8], by using a set of post-processing rules on top of the base NLU's slot-filling model. The rules were built on heuristics from the base NLU model, such as confidence score threshold, out-of-vocabulary flag, and span-expansion based on syntactic dependency tree.…”

Section: Nlu Gap Predictionmentioning

confidence: 99%

See 3 more Smart Citations

Interactive Teaching for Conversational AI

Ping¹,

Niu²,

Thattai³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Current conversational AI systems aim to understand a set of pre-designed requests and execute related actions, which limits them to evolve naturally and adapt based on human interactions. Motivated by how children learn their first language interacting with adults, this paper describes a new Teachable AI system that is capable of learning new language nuggets called concepts, directly from end users using live interactive teaching sessions. The proposed setup uses three models to: a) Identify gaps in understanding automatically during live conversational interactions, b) Learn the respective interpretations of such unknown concepts from live interactions with users, and c) Manage a classroom sub-dialogue specifically tailored for interactive teaching sessions. We propose state-of-the-art transformer based neural architectures of models, fine-tuned on top of pre-trained models, and show accuracy improvements on the respective components. We demonstrate that this method is very promising in leading way to build more adaptive and personalized language understanding models.

show abstract

Section: Definition Understanding Resultsmentioning

confidence: 82%

Section: Experiments and Resultsmentioning

confidence: 99%

Section: Definition Understanding Resultsmentioning

confidence: 99%

Section: Nlu Gap Predictionmentioning

confidence: 99%

See 2 more Smart Citations

Interactive Teaching for Conversational AI

Ping¹,

Niu²,

Thattai³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…They argue that developers cannot anticipate all the actions that users want, and that the system cannot understand the corresponding natural language even if the desired action is built-in. Like Jia et al (2017), Azaria et al (2016) starts with an ad-hoc set of initial slot-filling commands in natural language as the basis of further instructions-our approach starts with a more expressive core PL designed to interpolate with natural language. Compared to previous work, this work studied interactive learning in a shared community setting and hierarchical definitions resulting in more complex concepts.…”

Section: Related Work and Discussionmentioning

confidence: 99%

Naturalizing a Programming Language via Interactive Learning

Wang¹,

Ginn²,

Liang³

et al. 2017

Proceedings of the 55th Annual Meeting of the Association For Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

Our goal is to create a convenient natural language interface for performing wellspecified but complex actions such as analyzing data, manipulating text, and querying databases. However, existing natural language interfaces for such tasks are quite primitive compared to the power one wields with a programming language. To bridge this gap, we start with a core programming language and allow users to "naturalize" the core language incrementally by defining alternative, more natural syntax and increasingly complex concepts in terms of compositions of simpler ones. In a voxel world, we show that a community of users can simultaneously teach a common system a diverse language and use it to build hundreds of complex voxel structures. Over the course of three days, these users went from using only the core language to using the naturalized language in 85.9% of the last 10K utterances.

show abstract

Offline and Online Satisfaction Prediction in Open-Domain Conversational Systems

Choi

Ahmadvand

Agichtein

2019

Proceedings of the 28th ACM International Conference on Information and Knowledge Management

View full text Add to dashboard Cite

Predicting user satisfaction in conversational systems has become critical, as spoken conversational assistants operate in increasingly complex domains. Online satisfaction prediction (i.e., predicting satisfaction of the user with the system after each turn) could be used as a new proxy for implicit user feedback, and offers promising opportunities to create more responsive and effective conversational agents, which adapt to the user's engagement with the agent. To accomplish this goal, we propose a conversational satisfaction prediction model specifically designed for open-domain spoken conversational agents, called ConvSAT. To operate robustly across domains, ConvSAT aggregates multiple representations of the conversation, namely the conversation history, utterance and response content, and system-and user-oriented behavioral signals. We first calibrate ConvSAT performance against state of the art methods on a standard dataset (Dialogue Breakdown Detection Challenge) in an online regime, and then evaluate ConvSAT on a large dataset of conversations with real users, collected as part of the Alexa Prize competition. Our experimental results show that ConvSAT significantly improves satisfaction prediction for both offline and online setting on both datasets, compared to the previously reported stateof-the-art approaches. The insights from our study can enable more intelligent conversational systems, which could adapt in real-time to the inferred user satisfaction and engagement.

show abstract

Learning concepts through conversations in spoken dialogue systems

Cited by 6 publications

References 13 publications

Interactive Teaching for Conversational AI

Interactive Teaching for Conversational AI

Naturalizing a Programming Language via Interactive Learning

Offline and Online Satisfaction Prediction in Open-Domain Conversational Systems

Contact Info

Product

Resources

About