Motion and Sound in Animated Storybooks for Preschoolers’ Visual Attention and Mandarin Language Learning: An Eye-Tracking Study With Bilingual Children

Abstract

This study examined the role of the “animated eBook advantage” in child bilingual’s Mandarin learning, which has tended to be examined in the acquisition of Germanic languages. With this aim, 102 4- to 5-year-old preschoolers in Singapore were assigned to one of four conditions: (a) animated eBooks (+sound+motion), (b) static eBooks with sound, (c) static eBooks only, and (d) a control condition where children played a math game on an iPad. Three stories were displayed to children each for four times over 2 weeks, while visual attention was traced with an eye tracker. Children’s target words and story comprehension were assessed for the effects of the intervention conditions. The results revealed that children in the animated condition outperform their counterparts in total fixation duration, target word production, and storytelling of one of the stories (Cycling With Grandpa). There were no consistent differences between the two static conditions. Our results indicate the importance of motion in animated eBook design, in line with previous findings.

Vocabulary Development in Early Language Education

“…Shared book reading. It has been widely used in early language education and is considered one of the most effective approaches to help preschoolers build vocabulary (De Temple and Snow 2003;Sun et al 2019;Sun 2019). In the bilingual setting, the employment of dual language books (Sneddon 2008) and the presence of books in children's L1 in the classroom (Ernst-Slavit and Mulhern 2003) have been found to be effective language-teaching strategies in particular.…”

Multimedia Input and Bilingual Children’s Language Learning

“…They may fail to derive the meaning of unknown words and grammar from the verbal context and consequently have difficulties in figuring out the story plots (Verhallen and Bus, 2010). Well-designed animated eBooks hold good promise for children's emergent literacy in this case, as such books can stimulate readers' visual, auditory and even kinesthetic senses to comprehend a story and unfamiliar language via the congruence between non-verbal sources (motion pictures, images, sound, and music) and the narration (Sun et al, 2019), as predicted by the dual coding theory of learning. The "enhanced" message could scaffold learners to pick up the target information more easily and establish a coherent mental representation.…”

“…The rapid emergence of multimedia such as content delivered through computers, tablets, and electronic books has substantially reshaped bilingual children’s input environment ( Sun et al, 2019 ). Thus, the conventional input assessment that addresses children’s active communication with other interlocutors might not be able to holistically capture children’s daily input patterns.…”

Digital technology use and early reading abilities among bilingual children in Singapore

“…Moreover, adult-shared reading experiences using physical books have a larger positive impact on children’s story comprehension than using electronic books (Yuan, 2014). Recently, Sun et al. (2019) found that, as compared to static eBooks and no e-reading experiences, reading animated eBooks with motion and sound would significantly promote Singaporean children’s word production and storytelling abilities within a short period of time (two weeks).…”