There’s more to the multimedia effect than meets the eye: is seeing pictures believing?

Abstract

Textbooks in applied mathematics often use graphs to explain the meaning of formulae, even though their benefit is still not fully explored. To test processes underlying this assumed multimedia effect we collected performance scores, eye movements, and think-aloud protocols from students solving problems in vector calculus with and without graphs. Results showed no overall multimedia effect, but instead an effect to confirm statements that were accompanied by graphs, irrespective of whether these statements were true or false. Eye movement and verbal data shed light on this surprising finding. Students looked proportionally less at the text and the problem statement when a graph was present. Moreover, they experienced more mental effort with the graph, as indicated by more silent pauses in thinking aloud. Hence, students actively processed the graphs. This, however, was not sufficient. Further analysis revealed that the more students looked at the statement, the better they performed. Thus, in the multimedia condition the graph drew students' attention and cognitive capacities away from focusing on the statement. A good alternative strategy in the multimedia condition was to frequently look between graph and problem statement, and thus to integrate their information. In conclusion, graphs influence where students look and what they process, and may even mislead them into believing

Eye tracking in Educational Science: Theoretical frameworks and research agendas

“…In another example, we investigated the multimedia effect (Ögren, Nyström, & Jarodzka, 2016). In this study, we used multimedia material on the topic of vector calcu-lus.…”

“…Adding an eye movement display on top of that, is likely to overwhelm them. An alternative is to display the model's eye movements by reducing existing information on videos (Dorr, Vig, Gegenfurtner, Martinetz, & Barth, 2008;Nyström & Holmqvist, 2008). This results in a spotlight wandering across the video, while the rest of it appears blurred.…”

“…In the latter example presented above [ 61 ], we saw that the guidelines given, might need to be specified. Hence, it is crucial to test also the other guidelines for Instructional Design with eye tracking, but also under ecologically valid circumstances (i.e., actual learning material with real students).…”

Eye-tracking methodology in mathematics education research: A systematic literature review

“…In general, if data types were analyzed in relation to each other, it was often the relation between eye movements and accuracy that was analyzed. Richer triangulations were done with gestures and communication (Hannula & Williams, 2016;Shvarts, 2018aShvarts, , 2018b, interviews and stimulated recall (Klein et al, 2018;; see also , think-aloud protocols and selfreports (Cimen & Campbell, 2012;Green et al, 2007;Ögren et al, 2017;, cognitive load (Lin & Lin, 2014b), affective variables (Hunt et al, 2015;Strohmaier et al, 2017), or skin conductance and EEG (Muldner & Burleson, 2015).…”

“…In addition, 25% of the reviewed studies analyzed attentional patterns of eye movements, which are often associated with solution strategies. For example, the number and order of transitions between certain aspects of a stimulus was used to assess parallel compared with sequential strategies in number processing (Merkley & Ansari, 2010;Meyerhoff et al, 2012), fraction comparison strategies (Miller Singley & Bunge, 2018;Obersteiner & Tumpek, 2016), or information integration processes (Alqassab et al, 2018;Crisp et al, 2011;Ögren et al, 2017). As another measure, the position of the first fixation was considered an indicator for a preferred order of information processing (Michal et al, 2016).…”