Non-speech input and speech recognition for real-time control of computer games

Sporka, Adam J.; Kurniawan, Sri; Mahmud, Murni; Slavik, Pavel

doi:10.1145/1168987.1169023

Cited by 46 publications

(34 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The second task contained gestures used in the Tetris game controlled by humming [7]. The participants were again told to analyze the VGT expression: The expression above describes the templates depicted in Figure 5b.…”

Section: Task #2mentioning

confidence: 99%

“…The NVVI is an interaction method that has already received a significant focus within the research community. It has been used as an input modality for people with motor disabilities [7][3] as well as voice training tool [2]. It is a method that shares some similarities with Automatic Speech Recognition (ASR).…”

Section: Introductionmentioning

confidence: 99%

“…However, when comparing both interaction styles, several differences are revealed. Several reports, including mouse emulation [1] or controlling real-time games [7], suggest that NVVI is better fitted to continuous control rather than ASR. NVVI is cross-cultural and language independent [8].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Understanding Formal Description of Pitch-Based Input

Poláček¹,

Mikovec²

2010

Human-Centred Software Engineering

View full text Add to dashboard Cite

Abstract. The pitch-based input (humming, whistling, singing) in acoustic modality has already been studied in several projects. There is also a formal description of the pitch-based input which can be used by designers to define user control of an application. However, as we discuss in this paper, the formal description can contain semantic errors. The aim of this paper is to validate the formal description with designers. We present a tool that is capable of visualizing vocal commands and detecting semantic errors automatically. We have conducted a user study that brings preliminary results on comprehension of the formal description by designers and ability to identify and remove syntactic errors.

show abstract

Section: Task #2mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Understanding Formal Description of Pitch-Based Input

Poláček¹,

Mikovec²

2010

Human-Centred Software Engineering

View full text Add to dashboard Cite

show abstract

“…The analysis of humming (i.e. producing 'mmm' sound) was implemented as a user test [21] proved that humming was easier to produce and less tiresome for the users. The system runs as a standalone application on win32 platforms (see Fig.…”

Section: Description Of the Systemsmentioning

confidence: 99%

A Comparative Longitudinal Study of Non-verbal Mouse Pointer

Mahmud¹,

Sporka²,

Kurniawan³

et al. 2007

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract.A longitudinal study of two non-speech continuous cursor control systems is presented in this paper: Whistling User Interface (U 3 I) and Vocal Joystick (VJ). This study combines the quantitative and qualitative methods to get a better understanding of novice users' experience over time. Three hypotheses were tested in this study. The quantitative data show that U 3 I performed better in error rate and in simulating a mouse click; VJ was better on other measures. The qualitative data indicate that the participants' opinions regarding both tools improved day-by-day. U 3 I was perceived as less fatiguing than VJ. U 3 I approached the performance of VJ at the end of the study period, indicating that these two systems can achieve similar performances as users get more experienced in using them. This study supports two hypotheses but does not provide enough evidence to support one hypothesis.

show abstract

“…Slow, clunky, and intermittent connections may be acceptable with pseudodialog applications such as the ones typical in virtual assistants (Suendermann-Oeft, 2013), but they are not suited for realistic conversational applications such as in customer care (Acomb et al., 2007), virtual tutoring (Litman andSilliman, 2004), or command and control. Even more importantly, there are numerous applications for spoken dialog systems which require operation in offline mode altogether, for example in moving vehicles (Pellom et al, 2001), with robots in adverse conditions (Toptsis et al, 2004), in certain medical devices , or with portable video game consoles and toys (Sporka et al, 2006).Maintaining a cloud application server farm, capable of supporting the mass service comes at a recurring operational cost, which limits the range of possible revenue models with which the spoken dialog system can be offered. A way to solve this problem is to transfer the necessary hardware to the client device and let the customer naturally cover the processing power costs.…”

mentioning

confidence: 99%

Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue

Katagiri

Nakano

Fernández

et al. 2016

View full text Add to dashboard Cite

We extend special thanks to our Local co-Chairs, Ron Artstein and Alesia Gainer, and their team of student volunteers. We know SIGDIAL 2016 would not have been possible without Ron and Alesia, who invested so much effort in arranging the conference venue and accommodations, handling registration, making banquet arrangements, and handling numerous other preparations for the conference. The student volunteers for on-site assistance also deserve our appreciation.Ethan Selfridge, Sponsorships Chair, has earned our appreciation for recruiting and liaising with our conference sponsors, many of whom continue to contribute year after year. Sponsorships support valuable aspects of the program, such as the invited speakers and conference banquet. In recognition of this, we gratefully acknowledge the support of our sponsors: (Platinum level) Microsoft Research, Xerox and PARC, Intel, (Gold level) Facebook, (Silver level) Amazon Alexa, Interactions, Educational Testing Service, Honda Research Institute, and Yahoo!. At the same time, we thank Priscilla Rasmussen at the ACL for tirelessly handling the financial aspects of sponsorship for SIGDIAL 2016, and for securing our ISBN.iii We also thank the SIGdial board, especially officers Amanda Stent, Jason Williams and Kristiina Jokinen for their advice and support from beginning to end.Finally, we thank all the authors of the papers in this volume, and all the conference participants for making this stimulating event a valuable opportunity for growth in the research areas of discourse and dialogue. AbstractThis paper presents an end-to-end framework for task-oriented dialog systems using a variant of Deep Recurrent QNetworks (DRQN). The model is able to interface with a relational database and jointly learn policies for both language understanding and dialog strategy. Moreover, we propose a hybrid algorithm that combines the strength of reinforcement learning and supervised learning to achieve faster learning speed. We evaluated the proposed model on a 20 Question Game conversational game simulator. Results show that the proposed method outperforms the modular-based baseline and learns a distributed representation of the latent dialog state. IntroductionTask-oriented dialog systems have been an important branch of spoken dialog system (SDS) research (Raux et al., 2005; Young, 2006; Bohus and Rudnicky, 2003). The SDS agent has to achieve some predefined targets (e.g. booking a flight) through natural language interaction with the users. The typical structure of a task-oriented dialog system is outlined in Figure 1 (Young, 2006). This pipeline consists of several independently-developed modules: natural language understanding (the NLU) maps the user utterances to some semantic representation. This information is further processed by the dialog state tracker (DST), which accumulates the input of the turn along with the dialog history. The DST outputs the current dialog state and the dialog policy selects the next system action based on the dialog state. Then natural language gene...

show abstract

Non-speech input and speech recognition for real-time control of computer games

Cited by 46 publications

References 11 publications

Understanding Formal Description of Pitch-Based Input

Understanding Formal Description of Pitch-Based Input

A Comparative Longitudinal Study of Non-verbal Mouse Pointer

Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue

Contact Info

Product

Resources

About