Full Body Gestures Enhancing a Game Book for Interactive Story Telling

Abstract

Game Books can offer a well-written, but non-linear story, as readers always have to decide, how to continue after reading a text passage. It seems very logical to adopt such a book to investigate interaction paradigms for an interactive storytelling scenario. Nevertheless, it is not easy to keep the player motivated during a long-winded narrated story until the next point of intervention is reached. In this paper we tested different methods of implementing the decision process in such a scenario using speech input and tested it with 26 participants during a two player scenario. This revealed that with an omitted on-screen prompt the application was less easy to use, but caused considerably more user interaction. We further added additional interactivity with so-called Quick Time Events (QTEs). In these events, the player has a limited amount of time to perform a specific action after a corresponding prompt appeares on screen. Different versions of QTEs were implemented using Full Body Tracking with Microsoft Kinect, and were tested with another 18 participants during a two player scenario. We found that Full Body Gestures were easier to perform and, in general, preferred to controlling a cursor with one hand and hitting buttons with it.

User-Defined Body Gestures for an Interactive Storytelling Scenario

“…During this process we obtained a taxonomy of full body gestures for our interaction set, user ratings and agreement scores for each in-game action, the time performances of all gesture candidates, and we finally integrated the gesture candidates in our applications using our open source full body interaction framework FUBI [7] . A first validation for FUBI according to accuracy and usability was already done with a different interactive storytelling scenario [8] that included different kinds of iconic gestures. We plan to conduct a similar study with the new scenario in order to provide a more complete validation, also with more abstract metaphorical gestures.…”

“…This in turn poses new challenges for the interaction designer. Various researchers have already started to integrate this new kind of interaction in their interactive storytelling system [8,1], but usually the gesture set for interacting with the system is chosen by the developers themselves according to their imagination and preferences. However, a gesture that is intuitive for the developers does not necessarily have to be intuitive for the majority of users.…”

KINterestTV - Towards Non–invasive Measure of User Interest While Watching TV

“…), we chose here the simplest approach: the descriptive method. F. Kistler (from Augsburg University, Germany) developed the FullBody Interaction Framework (FUBI) [18], an open-source framework that uses a Kinect-like RGBD sensor and that has been successfully used in many situations [20], [19], [16].…”

A model-based approach for gesture interfaces

“… In [79], the authors selected a set of gestures for developing a machine-learning recognizer based on a restricted set of features.  In [12], the authors propose a gestural interface for the remote control of a robot  In [67], the authors propose a set of gestures for controlling a Google Maps through gestures  In [74], the authors enhanced a book story telling application, providing the possibility to select different paths on the plot through a set of gestures. A user study demonstrated that the users prefer such selection mechanism if compared with pressing buttons.…”

“…Such kind of definition is exploited for instance in [74] and [33]. The second one is more recognition-oriented and tries to mimic the walking movements with less physical effort for the users.…”