Children's Interaction with Motion-Based Touchless Games

Abstract

Leveraging movement data to support children's learning is appealing and technically challenging. However, there is limited knowledge about exploiting the complete design potential of bodily interplay in learning games. We conducted an in-the-wild study with 8 children, with special educational needs, playing a language based educational motion-based touchless game. We collected children's interaction data (correctness and reaction time), and data regarding the different design elements (game settings) implemented in 90 game sessions. Our analysis shows that number of items on-screen, selection gestures, and time to select items, impact the effectiveness (correctness) and efficiency (reaction time) of the children. We highlight the value of interaction analytics and quantify the relationship between different game design elements and children's efficiency and effectiveness. Our findings help shape the future of learning research by emphasising the substantial benefits of collecting movement data during children's interaction with learning games. CCS CONCEPTS • Human-centered computing → Gestural input; Empirical studies in HCI; Empirical studies in interaction design; Empirical studies in accessibility; • Applied computing → Interactive learning environments.

A systematic survey on embodied cognition: 11 years of research in child–computer interaction

“…The results show that although there was mention of methodologies for measuring 'cognitive-affective state' as one of the categories, only one study explicitly addressed the cognitive-affective states of the children, making use of realtime psychological markers [105]. While it seemed hard to explicitly determine if a number of the studies were augmenting pre-existing knowledge or helping children form new knowledge, some studies addressed this.…”

From luxury to necessity: Progress of touchless interaction technology

“…One possibility is Augmented Reality(AR) which provides touchless interaction using leapmotion [ 61 ], Microsoft kinect [ 60 ], azure kinect [ 62 ] and latest SDKs as described in section 2.2 . It allows for embodied digital learning apps with motion and hand tracking cameras which enables interactions with 3D learning material like a remote laboratory [ 63 ] and allows for the avoiding of touching physical learning materials.…”

“…

Fig. 14 Use of touchless technology using kinect for children education [ 60 ] (ii) Using leapmotion touchless interaction hand for learning PC assembling [ 61 ].

…”

Investigating the Effect of Interface Type on Embodied Interactions in 3D Gestural Human-Computer Interaction

“…Through emerging technologies, it is possible to integrate embodiment into interacting with computers in the form of embodied human-computer interaction (eHCI). Embodied interactions have shown promise in several domains (Benford et al, 2021;Dworkin et al, 2018;Lee-Cultura et al, 2020;Sarma & Bhuyan, 2021;Spiel, 2021). Technologies such as augmented and virtual reality implement an eHCI style using virtual reality (VR) headsets or data gloves where people can now use their bodies to interact with computers, but these technologies are invasive to the end user as they are typically bulky equipment.…”