Scientific Visualizations Based on Integrated Model of Text and Picture Comprehension via Eye-tracking

Abstract

This study investigates the "color combination effect" and the "sequence of pictures and text effect" in the "Integrated Model of Text and Picture Comprehension" (ITPC) on the formation of the mental model and cognitive load in both desktop and mobile devices. These effects are studied using an experiment including scientific contexts with specific color combinations and different picture and text sequences. The participants of this experiment are graduate students from different fields of science in Universiti Sains Malaysia. Four tasks are assigned for each participant in each phase and every task includes a scientific context accompanied with visualization. After each task the participant is asked to answer a set of questions related to the context. Eye-tracking methods are adopted to record gaze data of the participants while reading each context. Analysis of the gaze data and the percentage of the correct answers to the questions indicate that the sequence of picture and text and use of color in the visualizations results in reduction of the cognitive load of the participants.

An eye-tracking study on the effect of infographic structures on viewer’s comprehension and cognitive load

“…In the second analysis, the overall increase in the cognitive load for each stimulus is calculated using the CLS (Cognitive Load Score) calculation methods. 31…”

“America First, the Netherlands Second” on YouTube: “spoofing” destination marketing with political satire

“…Optimizing user experience plays an important role in today's marketing research and practice, with the prime focus on visual attention of the viewers and its effects, such as memorability (Chen & Chen 2017;Krucien, Ryan & Hermens 2017). Majooni, Masood and Akhavan (2014) found that memorability was enhanced by the use of graphical content elements followed by textual content elements. Smith, Boerman and Meurs (2015) The most accurate approach to analysing visual attention is to use a gaze-tracking/eyetracking device.…”

Human-robot collaboration on an assembly work with extended cognition approach

“…Liu and Chuang (2010) and Liu et al (2011), showed the negative effects of redundant images and information. On the other hand, Majooni et al (2015) revealed with their experiment that when picture is presented before the text, it causes less cognitive load than the load in reverse sequence. One major point of the interface design is the interactivity of the interfaces, especially those for computerbased learning systems.…”

“…These tasks, interfaces and individuals always work in correlation and influence the cognitive load. Concerning this correlation and influence, usability and CLT studies have been conducted in the area of cognitive ergonomics (Ahlstrom and Freidman-Berg, 2006;Evans and Gibbons, 2007;Cierniak et al, 2008;Austin, 2009;van Gog et al, 2009a;De Koning et al, 2010;Liu and Chuang, 2010;Liu et al, 2011;Galy et al, 2012;Wang et al, 2014;van Cauwenberge et al, 2014;Huang and Chen, 2015;Jiang et al, 2015;Majooni et al, 2015;Vatavu and Mancas, 2015). Galy et al (2012) investigated the relationship between mental workload factors (task difficulty, time pressure and alertness) and cognitive load types.…”