Study on Gesture and Voice-based Interaction in Perspective of a Presentation Support Tool

Abstract

Objective: This study aims to implement a non-contact gesture-based interface for presentation purposes and to analyze the effect of the proposed interface as information transfer assisted device. Background: Recently, research on control device using gesture recognition or speech recognition is being conducted with rapid technological growth in UI/UX area and appearance of smart service products which requires a new human-machine interface. However, few quantitative researches on practical effects of the new interface type have been done relatively, while activities on system implementation are very popular. Method: The system presented in this study is implemented with KINECT ® sensor offered by Microsoft Corporation. To investigate whether the proposed system is effective as a presentation support tool or not, we conduct experiments by giving several lectures to 40 participants in both a traditional lecture room(keyboard-based presentation control) and a non-contact gesture-based lecture room(KINECT-based presentation control), evaluating their interests and immersion based on contents of the lecture and lecturing methods, and analyzing their understanding about contents of the lecture. Result: We check that whether the gesture-based presentation system can play effective role as presentation supporting tools or not depending on the level of difficulty of contents using ANOVA. Conclusion: We check that a non-contact gesture-based interface is a meaningful tool as a sportive device when delivering easy and simple information. However, the effect can vary with the contents and the level of difficulty of information provided. Application: The results presented in this paper might help to design a new human-machine(computer) interface for communication support tools.

Teachme, A Gesture Recognition System with Customization Feature

“…However, the lecturer still need to move back and forth between the computer and projection plan from one side and the students or the audiences for better communication from another side [2] and [3]. Additionally, using keyboard and mouse reduces the interactive and naturalness in communication with audiences [4] and [5]. References [2] and [1] introduce that even the Bluetooth connected devices, which are designed to control presentations have some drawbacks regarding the problem of keeping the lecturer close the computer because of its limited functionalities.…”