Avoiding split attention in computer‐based testing: Is neglecting additional information facilitative?

Abstract

This study investigated whether design guidelines for computer-based learning can be applied to computer-based testing (CBT). Twenty-two students completed a CBT exam with half of the questions presented in a split-screen format that was analogous to the original paper-and-pencil version and half in an integrated format. Results show that students attended to all information in the integrated format while ignoring information in the split format. Interestingly, and contrary to expectations, they worked more efficiently in the split format. A content analysis of the ignored information revealed that it was mostly not relevant to answering the questions, unnecessarily taxed students' cognitive capacity and inefficiently increased the mental effort they expended. Further comparisons of different mental effort measures indicate that mental effort had an explicit (ie, self-reports, explicit utterances) and an implicit component (ie, silent pauses in thinking-aloud, eye tracking parameters). Consequently, when designing CBT environments, not only the design of the tasks but also the content of the given information and their effect on the different aspects of mental effort must be considered. IntroductionAs technology use in learning increases, the use of technology for assessment is also becoming more common. In the Netherlands, for example, computer-based testing (CBT) is making its way into the national exams used in secondary education (De Boer, 2009) by the Dutch Institute of Educational Measurement (CITO; www.cito.nl) that develops standardized tests for schools on behalf of the government. CBT provides the possibility to include a mix of different presentation formats, such as videos, text, pictures, etc. (ie, multimedia: Mayer, 2005a), which is increasingly being implemented (cf. Parshall, Harmes, Davey & Pashley, 2010). Researchers, however, have emphasized that it is important to appropriately design CBT environments so that students can focus on test content and are not impeded by difficulties relating to its design. For instance, Parshall, Spray, Kalohn and Davey (2002, p 5) stated that ". . . examinees need to know clearly to what part of the screen they must attend to, how to navigate, and how to indicate a response.

There’s more to the multimedia effect than meets the eye: is seeing pictures believing?

“…Sweller et al (1998) proposed that working memory capacity can be filled with three types of loads, namely load caused by active processing of the information (germane load), load stemming from the difficulty of the task (intrinsic load), and by load stemming from other unnecessary cognitive processes that do not contribute to executing the task at hand (extraneous load). The amount of cognitive load posed upon working memory (i.e., mental effort) can be measured with different methods, such as subjective rating scales (Paas, 1992) or silent pauses in thinking aloud (Yin & Chen, 2007;Jarodzka, Janssen, Kirschner, & Erkens, 2015).…”

Before your very eyes: the value and limitations of eye tracking in medical education

“…printed text and pictures) is presented close together, so learners do not have to visually search for information in order to be able to integrate it. Interestingly, Jarodzka et al 27 did not find a positive effect of contiguity in a computer-based testing situation. Eye tracking was used to understand these findings.…”

Gaze-Driven Design Insights to Amplify Debugging Skills: A Learner-Centered Analysis Approach

“…In these learning scenarios, eye-tracking data can aid researchers and educators in studying attention and shifts in attention underlying the split-attention effect. Past research has shown that under experimental conditions, learners focus on text; under naturalistic conditions, they focus on images, and when the information is appropriately integrated, then learners process it all (Jarodzka, Janssen, Kirschner, & Erkens, 2015). Moreover, eye-tracking data has shown that as learners increase their knowledge and expertise they tend to fixate faster on more relevant information Jarodzka, Scheiter, Gerjets, & Van Gog, 2010).…”