Experiences of an in-service Wizard-of-Oz data collection for the deployment of a call-routing application

Wirén, Mats; Eklund, Robert

doi:10.3115/1556328.1556336

Cited by 10 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multiple methods have been proposed in using realistic data for developing chatbots. Wirén et al (2007) suggest a modified version of the wizard of the oz approach by collecting transcripts of real conversations between service agents and customers. Many bots are being built to augment human service agents, and hence there is a rich set of information available as human (customer) -human (service agent) conversations.The tool Graph2Bot (Bouraoui et al, 2019) analyzes such existing conversations but fails to create a format that can be leveraged by commercial tools.…”

Section: Related Workmentioning

confidence: 99%

ICM : Intent and Conversational Mining from Conversation Logs

Mitra,

Ramnani,

Ranjan

et al. 2022

Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue

View full text Add to dashboard Cite

Building conversation agents requires considerable manual effort in creating training data for intents / entities as well as mapping out extensive conversation flows. In this demonstration, we present ICM (Intent and Conversation Mining), a tool which can make the BOT build and update process much faster. ICM can be used to analyze existing conversation logs and help a bot designer to cluster, visualize and analyze customer intents; train custom intent models; and also to map and optimize conversation flows. The tool can be used for first time deployment or subsequent conversational flow updates in chatbots.

show abstract

Section: Related Workmentioning

confidence: 99%

ICM : Intent and Conversational Mining from Conversation Logs

Mitra,

Ramnani,

Ranjan

et al. 2022

Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue

View full text Add to dashboard Cite

show abstract

“…We have not yet adopted this metric for evaluation of dialogues from the live system. However, elsewhere we have applied it to dialogues from the initial Wizard-of-Oz data collection for the Telia-Sonera call routing system (Wirén et al 2007).…”

Section: Dialogue-based Metricsmentioning

confidence: 99%

Multi-slot semantics for natural-language call routing systems

Boye

Wirén

2007

Proceedings of the Workshop on Bridging the Gap Academic and Industrial Research in Dialog Technologies - NAACL-HLT '07

Self Cite

View full text Add to dashboard Cite

Statistical classification techniques for natural-language call routing systems have matured to the point where it is possible to distinguish between several hundreds of semantic categories with an accuracy that is sufficient for commercial deployments. For category sets of this size, the problem of maintaining consistency among manually tagged utterances becomes limiting, as lack of consistency in the training data will degrade performance of the classifier. It is thus essential that the set of categories be structured in a way that alleviates this problem, and enables consistency to be preserved as the domain keeps changing. In this paper, we describe our experiences of using a twolevel multi-slot semantics as a way of meeting this problem. Furthermore, we explore the ramifications of the approach with respect to classification, evaluation and dialogue design for call routing systems.

show abstract

“…By using a human 'wizard' to mimic the functionality of a system, either completely or in part, WOZ supports the evaluation of potential user experiences and interaction strategies without the need for building a fully functional product first (Gould et al, 1983). It furthermore supports the collection of domain specific language corpora and the easy exploration of varying dialog designs (Wirén et al, 2007). WOZ tools, however, are often application dependent and built for very specific experimental setups.…”

Section: Introductionmentioning

confidence: 99%

Proceedings of the Demonstrations at the 14th Conference of the European Chapter of the Association for Computational Linguistics

Basile

Bos

Evang

et al. 2014

View full text Add to dashboard Cite

Welcome to EACL 2014, the 14th Conference of the European Chapter of the Association for Computational Linguistics! This is the largest EACL meeting ever: with eighty long papers, almost fifty short ones, thirteen student research papers, twenty-six demos, fourteen workshops and six tutorials, we expect to bring to Gothenburg up to five hundred participants, for a week of excellent science interspersed with entertaining social events.It is hard to imagine how much work is involved in the preparation of such an event. It takes about three years, from the day the EACL board starts discussing the location and nominating the chairs, until the final details of the budget are resolved. The number of people involved is also huge, and I was fortunate to work with an excellent, dedicated and efficient team, to which I am enormously grateful.The scientific program was very ably composed by the Program Committee Chairs, Sharon Goldwater and Stefan Riezler, presiding over a team of twenty-four area chairs. Given that this year we had long paper submissions, followed by a rebuttal period, followed by a very stressed short paper reviewing period, this meant a lot of work. Overall, Sharon and Stefan handled over five hundred submissions, or over 1,500 reviews! The result of this work is a balanced, high-quality scientific program that I'm sure we will all enjoy. The PC Chairs have also selected the three invited speakers, and we will have the pleasure of attending keynotes delivered by Simon King, Ulrike von Luxburg, and Dan Roth -a great choice of speakers! The diverse workshop program was put together by the Workshop Chairs, Anja Belz and Reut Tsarfaty, under very strict deadlines due to the fact that as in previous years, workshops were coordinated with other ACL events (this year, ACL and EMNLP). Even in light of the competition, Anja and Reut negotiated a varied and attractive set of fourteen workshops which will keep us busy over the weekend prior to the main conference.Also on that weekend are the six tutorials, selected from among several submissions by the Tutorial Chairs, Afra Alishahi and Marco Baroni. Again, the tutorials offer a set of diverse and timely topics, covering both core areas of NLP and tangential fields of research.

show abstract

Experiences of an in-service Wizard-of-Oz data collection for the deployment of a call-routing application

Cited by 10 publications

References 11 publications

ICM : Intent and Conversational Mining from Conversation Logs

ICM : Intent and Conversational Mining from Conversation Logs

Multi-slot semantics for natural-language call routing systems

Proceedings of the Demonstrations at the 14th Conference of the European Chapter of the Association for Computational Linguistics

Contact Info

Product

Resources

About