Interaction Quality: Assessing the quality of ongoing spoken dialog interaction by experts—And how it relates to user satisfaction

Schmitt, Alexander; Ultes, Stefan

doi:10.1016/j.specom.2015.06.003

Cited by 44 publications

(71 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, the DM features yield better results than the ASR features and thus contribute more to the overall IQ value, 2 The arithmetic average of all class-wise recalls. which is in line with the outcomes of previous work (Ultes et al, 2015). It is stressed that none of the feature sets employed for the LSTM uses handcrafted temporal features nor needs them.…”

supporting

confidence: 87%

“…IQ is a more objective approach to US that relies on the rating of experts instead of users (Schmitt and Ultes, 2015) and thus closes the gap between subjective valuation and objective criteria. The respective rating is given on a scale between 1 (extremely unsatisfied) and five (satisfied) after listening to audio records of the dialogue in question.…”

Section: Introductionmentioning

confidence: 99%

“…Although the results were above the random baseline, the respective improvement was only minor. As it was discussed by Higashinaka et al (2010b), one difficulty of this task lies in the subjective nature of US since it depends on the appreciation of the user.IQ is a more objective approach to US that relies on the rating of experts instead of users (Schmitt and Ultes, 2015) and thus closes the gap between subjective valuation and objective criteria. The respective rating is given on a scale between 1 (extremely unsatisfied) and five (satisfied) after listening to audio records of the dialogue in question.…”

mentioning

confidence: 99%

“…In web search and dialogue systems, reformulation is known as an implicit feedback signal that the user could not receive a desired response to the previous input due to one or more system components failing. In IAs, ASR error is a major cause of reformulation and has been extensively studied Schmitt and Ultes, 2015).Besides correcting ASR errors, users of IAs reformulate their previous utterances when they encounter NLU errors, LG errors, and so on. For example, when a user utters "alarm", the NLU component may mistakenly conclude that s/he wants to perform a Web search, and consequently the system shows the search results for "alarm."…”

mentioning

confidence: 99%

See 3 more Smart Citations

Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue

2017

View full text Add to dashboard Cite

We are excited to welcome you to this year's SIGdial Conference, the 18th Annual Meeting of the Special Interest Group on Discourse and Dialogue. We are pleased to hold the conference in Saarbrücken on August 15-17th, co-located with SemDial 2017, and in close proximity to both INTERSPEECH 2017 and YRRSDS 2017, the Young Researchers' Roundtable on Spoken Dialog Systems.The SIGdial conference remains a premier publication venue for research in discourse and dialogue. This year, the program includes oral presentations, poster sessions, and one demo session. SIGdial 2017 also hosts three special sessions, two joint with SemDial 2017.We received 113 submissions this year. In only one previous year has there been a greater number of submissions to SIGdial. All long and short papers received at least 3 reviews. We carefully considered both the numeric ratings and the tenor of the comments in making our selections for the program. Overall, the members of the Program Committee did an excellent job in reviewing the submitted papers. We thank them for their important role in selecting the accepted papers and for helping to come up with a high quality program for the conference. We also thank Pierre Lison, Mentoring Chair for SIGdial 2017, for his dedicated work on the mentoring process. The goal of mentoring is to assist authors of papers that contain important ideas but lack clarity. In line with the SIGdial tradition, our aim has been to create a balanced program that accommodates as many favorably rated papers as possible. We accepted 29 long papers, 11 short papers and 7 demo presentations. These numbers give an overall acceptance rate of 41.6%. The rates separately for types of papers are 42% for long papers, 30% for short and 87% for demo submissions. Of the long papers, 18 were presented as oral presentations. The remaining 11 long papers and all the short papers were presented as posters, split into two poster sessions.This year SIGdial has three special sessions on topics of growing interest. The chosen sessions were (i) the Special Session on Negotiation Dialog organized by Amanda Stent, Aasish Pappu, Diane Litman and Marilyn Walker; (ii) the Second WOCHAT Special Session on Chatbots and Conversational Agents organized by Ryuichiro Higashinaka, Ron Artstein, Rafael E. Banchs, and Wolfgang Minker; and (iii) Special Session on Natural Language Generation for Dialog Systems organized by Marilyn Walker, Verena Rieser, Vera Demberg, Dietrich Klakow, Dilek Hakkani-Tur, David M. Howcroft and Shereen Oraby. These specialized topics brought diverse paper submissions to our technical program. At the conference, the special sessions also featured panel discussions and position talks, allowing for active engagement of the conference participants. This year, two of these special sessions, the one on negotiation and on conversational agents, are part of the joint SIGdial/SemDial program at the conference bringing both communities to participate in them. This year's SIGdial conference runs 3 full days compared to previous year...

show abstract

supporting

confidence: 87%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue

2017

View full text Add to dashboard Cite

show abstract

“…In the area of spoken dialogue systems, signals recognised from linguistic cues and prosody have been used to detect problematic dialogues (Herm et al, 2008) and to assess dialogue quality as a whole (Schmitt and Ultes, 2015). This type of dialogue-related signals has also been used to automatically detect miscommunication (Meena et al, 2015), or to predict the user satisfaction (Schmitt et al, 2011).…”

Section: Introductionmentioning

confidence: 99%

Proceedings of the First Workshop on Language Grounding for Robotics

2017

View full text Add to dashboard Cite

ii IntroductionAfter the remarkable successes of recent work visually grounded models of language, the embodied and task-oriented aspects of language learning stand as a natural next challenge. As autonomous robotic agents become increasingly capable and are deployed to progressively more complex environments, expressive, accessible interfaces are becoming essential to realizing the potential of such technologies. Natural language is immediately available to non-expert users and expressive enough to represent complex actions and plans. Can we give instructions to robotic agents to assist with navigation and manipulation tasks in remote settings? Can we talk to robots about the surrounding visual world, and help them interactively learn the language needed to finish a task? To build robots that we can converse with in our homes, offices, hospitals, and warehouses, it is essential that we develop new techniques for linking language to action in the real world.While the opportunity is clear, enabling effective interaction between users and autonomous agents requires addressing some of the core open challenges in NLP while studying new domains and tasks. This workshop aims to explore these challenges, bringing together members of the NLP, robotics, and vision communities to focus on language grounding in robots and other interactive goal-driven systems. The program features twelve new articles and seven cross-submissions from related areas, to be presented as both posters and talks. We are also excited to host remarkable invited speakers, including Regina Barzilay, Joyce Chai, Karl Moritz Hermann, Hadas Kress-Gazit, Terence Langendoen, Percy Liang, Ray Mooney, Nicholas Roy, Stefanie Tellex and Jason Weston.We thank the program committee, the ACL workshop chairs Wei Xu and Jonathan Berant, the invited speakers, and our sponsors DeepMind and Facebook.

show abstract

Voice Interaction for Training: Opportunities, Challenges, and Recommendations from HCI Perspective

Kondratova

Emond

2020

Learning and Collaboration Technologies. Human and Technology Ecosystems

View full text Add to dashboard Cite

Interaction Quality: Assessing the quality of ongoing spoken dialog interaction by experts—And how it relates to user satisfaction

Cited by 44 publications

References 37 publications

Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue

Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue

Proceedings of the First Workshop on Language Grounding for Robotics

Voice Interaction for Training: Opportunities, Challenges, and Recommendations from HCI Perspective

Contact Info

Product

Resources

About