Adding Physical Objects to an Interactive Game Improves Learning and Enjoyment

Abstract

Can experimenting with three-dimensional (3D) physical objects in mixed-reality environments produce better learning and enjoyment than flat-screen two-dimensional (2D) interaction? We explored this question with EarthShake: a mixed-reality game bridging physical and virtual worlds via depth-camera sensing, designed to help children learn basic physics principles. In this paper, we report on a controlled experiment with 67 children, 4-8 years old, that examines the effect of observing physical phenomena and collaboration (pairs vs. solo). A follow-up experiment with 92 children tests whether adding simple physical control, such as shaking a tablet, improves learning and enjoyment. Our results indicate that observing physical phenomena in the context of a mixed-reality game leads to significantly more learning and enjoyment compared to screen-only versions. However, there were no significant effects of adding simple physical control or having students play in pairs vs. alone. These results and our gesture analysis provide evidence that children's science learning can be enhanced through experiencing physical phenomena in a mixed-reality environment.

Learning from Mixed-Reality Games

“…Yannier et al showed that children used more meaningful gestures while explaining their predictions when they observed physical blocks in a mixed-reality game, compared to a screen-only version of the same game. This result suggests that children may be having more meaningful mental visualizations (demonstrated by the gestures they use) when they interact with 3D objects [30].…”

“…In a recent study, Yannier et al showed that children observing physical towers in a mixed-reality game learned physics principles better and showed higher levels of enjoyment compared to students interacting with a solely virtual game on a laptop [30]. This result suggests that observing physical phenomena (along with mixed reality feedback on whether child predictions and physical results match) may enhance learning and enjoyment.…”

“…Utilizing computer-vision technology to provide welldesigned interactive feedback to children while they interact with physical objects in their everyday environment may be an engaging and powerful way to learn [30,17]. However one can question whether real world interaction is necessary, especially given that interactive responsiveness and feedback to real world actions is more difficult to implement.…”

Empowering CH Experts to Produce IoT-enhanced Visits

“…Visitors of CH sites can touch, manipulate and interact with smart objects, for example to receive personalized information during the visit. Such interaction fosters the appropriation of CH content [7]: as recognized by some recent works [9], physical manipulation is indeed an effective supplementary channel for the visitor to gather and store information, since they are enabled to activate real-world knowledge.…”

“…End-User Development (EUD) methodologies fit very well the requirement of letting end users, with or without expertise in computer programming, to tailor, customize and program their software [2,4,6] or ecologies of smart objects [9]. Our approach proposes a visual composition paradigm that facilitates the creation of Event-Condition-Action (ECA) rules required to synchronize the behavior of multiple objects [5].…”