Explaining the modality and contiguity effects: New insights from investigating students' viewing behaviour

Abstract

The study examined viewing behaviour and learning outcome during multimedia learning in order to explore split-attention processes in modality and spatial contiguity effects. Fourty students viewed a computer instruction depicting the process of lightning. Exploratory text was spoken, written near or written far from accompanying animations. Students who received spoken text outperformed students who received written text in recalling the major steps (retention) and in identifying correct solutions to problems (transfer), replicating a modality effect. Differences between near and far written text presentation in retention (d ¼ 0.51) and transfer (d ¼ 0.68) tests failed statistical significance. Two major characteristics concerning the learners' viewing behaviour were identified: (a) in written text presentation, learning was largely text directed and (b) learning success was related to the time learners' spent looking at animations, indicating that the processing of animations is a crucial factor in explanations of modality and spatial contiguity effects.A basic affordance for learners in a multimedia learning environment is to extract and integrate information from different sources of information like words and pictures. A common recommendation for instructional designers is (a) to present words near rather than far from corresponding pictures or, better yet, (b) to present those corresponding words as spoken rather than printed text (e.g. Mayer, 2001). These empirically wellsupported recommendations are known as (a) the spatial contiguity principle (for a recent review of contiguity effects, see Ginns, 2006) and (b) the modality principle (for a recent review of modality effects, see Ginns, 2005).For example, Moreno and Mayer (1999, Experiment 1) presented a sequence of 16 animated illustrations depicting the process of lightning. The animations visualized e.g. the motion of cool air that becomes heated or positive charges moving up to the cloud producing a flash light. Animations were accompanied by an expository text describing each of the major events. Text was spoken, written inside the visualization frame or written below the visualization frame. Participants performed better on subsequent retention and transfer tests when (a) text was spoken rather than written (modality effect) APPLIED COGNITIVE PSYCHOLOGY

Explaining the modality effect in multimedia learning: Is it due to a lack of temporal contiguity with written text and pictures?

“…Thus, with sequential presentation the written text group was able to process the pictorial information adequately, resulting in no modality effect for pictorial recall with sequential presentation. This explanation receives further support from eye tracking studies showing that in multimedia learning environments learners predominantly read the text, whereas picture processing is secondary (e.g., Schmidt-Weigand et al, 2010). Thus, in system-paced learning environments, where the speed of presentation is typically determined by the time it takes to present the spoken text, it might be that for written text presentations the next slide is presented before the learner has had the chance to adequately process the picture, thereby yielding an insufficient processing of the pictorial information.…”

“…Moreover, we added a picture-based retention test to complement the verbal, text-based recall measure (cf. Schmidt-Weigand, Kohnert, & Glowalla, 2010).…”

The Role of Working Memory in Multimedia Instruction: Is Working Memory Working During Learning from Text and Pictures?

“…Most importantly, the authors found no interference between written text and the spatial tapping task, which is not in line with the assumption that written text is initially processed in the visual/pictorial channel as proposed by the Cognitive Theory of Multimedia Learning and Cognitive Load Theory. For alternative explanations of the modality effect that are based on sensory memory rather than working memory processes, the reader is referred to Rummer et al (2010) and Schmidt-Weigand et al (2010.…”

How Inspecting a Picture Affects Processing of Text in Multimedia Learning

“…Learning with text and pictures thus appeared to be largely guided by the text in that the text base directs attention toward those parts of the picture that are addressed in the text. This idea of text-guided processing of pictures is well acknowledged in the literature on learning from text and pictures (e.g., Folker, Ritter, & Sichelschmidt, 2005;Hegarty & Just, 1993;Ozcelik, Arslan-Ari, & Cagiltay, 2010;Rummer, Schweppe, Fürstenberg, Scheiter, & Zindler, 2011;Schmidt-Weigand, Kohnert, & Glowalla, 2010a, 2010bSchwonke, Berthold, & zRenkl, 2009;Van Gog, Kester, Nievelstein, Giesbers, & Paas, 2009). …”