Assembling the Jigsaw: How Multiple Open Standards Are Synergistically Combined in the HALEF Multimodal Dialog System

Ramanarayanan, Vikram; Suendermann-Oeft, David; Lange, Patrick; Mundkowsky, Robert; Ivanov, Alexei V.; Yu, Zhou; Ye, Qian; Evanini, Keelan

doi:10.1007/978-3-319-42816-1_13

Cited by 19 publications

(11 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…HALEF is an open-source, modular, cloud-based dialog system that is compatible with multiple W3C and open industry standards. The HALEF architecture and components have been described in detail in prior publications [13,14,15].…”

Section: The Halef Dialog Ecosystemmentioning

confidence: 99%

Human and Automated Scoring of Fluency, Pronunciation and Intonation During Human–Machine Spoken Dialog Interactions

et al. 2017

Self Cite

View full text Add to dashboard Cite

We present a spoken dialog-based framework for the computerassisted language learning (CALL) of conversational English. In particular, we leveraged the open-source HALEF dialog framework to develop a job interview conversational application. We then used crowdsourcing to collect multiple interactions with the system from non-native English speakers. We analyzed human-rated scores of the recorded dialog data on three different scoring dimensions critical to the delivery of conversational English-fluency, pronunciation and intonation/stress-and further examined the efficacy of automatically-extracted, hand-curated speech features in predicting each of these subscores. Machine learning experiments showed that trained scoring models generally perform at par with the human inter-rater agreement baseline in predicting human-rated scores of conversational proficiency.

show abstract

Section: The Halef Dialog Ecosystemmentioning

confidence: 99%

Human and Automated Scoring of Fluency, Pronunciation and Intonation During Human–Machine Spoken Dialog Interactions

et al. 2017

Self Cite

View full text Add to dashboard Cite

show abstract

Section: Audiovisual Dialog Dataset Generationmentioning

confidence: 99%

Rushing to Judgement: How do Laypeople Rate Caller Engagement in Thin-Slice Videos of Human–Machine Dialog?

2017

Self Cite

View full text Add to dashboard Cite

We analyze the efficacy of a small crowd of naïve human raters in rating engagement during human-machine dialog interactions. Each rater viewed multiple 10 second, thin-slice videos of non-native English speakers interacting with a computerassisted language learning (CALL) system and rated how engaged and disengaged those callers were while interacting with the automated agent. We observe how the crowd's ratings compared to callers' self ratings of engagement, and further study how the distribution of these rating assignments vary as a function of whether the automated system or the caller was speaking. Finally, we discuss the potential applications and pitfalls of such a crowdsourced paradigm in designing, developing and analyzing engagement-aware dialog systems.

show abstract

“…The Can you guess who I am? interactive activity was developed in HALEF, an open-source, modular, web-based framework for designing and deploying SDS tasks [26]. 1…”

Section: Description Of the Activitymentioning

confidence: 99%

Can you guess who I am?: An interactive task for young learners to practice yes/no question formation in English

Timpe‐Laughlin¹,

Lee²,

Evanini³

et al. 2017

6th Workshop on Child Computer Interaction (WOCCI 2017)

Self Cite

View full text Add to dashboard Cite

Although yes/no questions are one of the most frequently occurring question types in English, research on the development and production of yes/no questions-in particular in young English learners-is still very limited. For example, we know very little about potential errors young L2 learners make when they produce yes/no questions-an area that is crucial in order to provide useful feedback in different learning environments, including computer-based applications. This paper reports on an exploratory study conducted with Can you guess who I am?, an interactive, spoken-dialogue-based speaking activity that allows young English learners to practice yes/no questions. After introducing the SDS-based speaking activity, we present the findings from a systematic investigation of the output produced by 27 young English learners in Germany (ages 9-11) who engaged with the activity. A particular focus in the analysis was placed on the types of yes/no questions elicited and the types of errors made by the young learners. The findings provide further empirical support for a six-stage framework for the development of question formation in L2 learners [14]. Moreover, they offer insights into the types of errors young EFL learners make in forming polar interrogatives such as systematic confusion with regard to the auxiliaries "to be" and "to do". The findings are discussed in terms of (a) how they contribute to a more comprehensive understanding of young learner's speech and (b) how they will be used to inform further development of more targeted feedback options that can be implemented into the SDSbased speaking activity in order to harness its full potential for L2 learning.

show abstract

Assembling the Jigsaw: How Multiple Open Standards Are Synergistically Combined in the HALEF Multimodal Dialog System

Cited by 19 publications

References 16 publications

Human and Automated Scoring of Fluency, Pronunciation and Intonation During Human–Machine Spoken Dialog Interactions

Human and Automated Scoring of Fluency, Pronunciation and Intonation During Human–Machine Spoken Dialog Interactions

Rushing to Judgement: How do Laypeople Rate Caller Engagement in Thin-Slice Videos of Human–Machine Dialog?

Can you guess who I am?: An interactive task for young learners to practice yes/no question formation in English

Contact Info

Product

Resources

About