Multimedia integration signals highlight correspondences between text and pictures with the aim of supporting learning from multimedia. A recent meta-analysis revealed that only learners with low domain-specific prior knowledge benefit from multimedia integration signals. To more thoroughly investigate the influence of prior knowledge on the multimedia signaling effect in a more ecologically valid context, we conducted a quasi-experimental field study with 8th graders. They learned with a digital multimedia textbook in 1 of the 2 versions: (a) a basic version with signals that supported only the selection and organization of information from either text or pictures or (b) an extended version with additional multimedia integration signals to support the integration of information from text and pictures (e.g., color coding, deictic references). Results of a contrast analysis revealed that low-prior-knowledge learners learned better with the extended version compared with the basic version, whereas adding multimedia integration signals was detrimental for learning outcomes of high-prior-knowledge learners. This expertise reversal effect could only partially be explained by cognitive load measures, in that high-prior-knowledge learners had higher extraneous cognitive load in the condition with multimedia integration signals. The results suggest a need for a more individualized multimedia design that considers students’ prior knowledge.
Modeling eye movement indicative of expertise behavior is decisive in user evaluation. However, it is indisputable that task semantics affect gaze behavior. We present a novel approach to gaze scanpath comparison that incorporates convolutional neural networks (CNN) to process scene information at the fixation level. Image patches linked to respective fixations are used as input for a CNN and the resulting feature vectors provide the temporal and spatial gaze information necessary for scanpath similarity comparison. We evaluated our proposed approach on gaze data from expert and novice dentists interpreting dental radiographs using a local alignment similarity score. Our approach was capable of distinguishing experts from novices with 93% accuracy while incorporating the image semantics. Moreover, our scanpath comparison using image patch features has the potential to incorporate task semantics from a variety of tasks.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.