Is a picture worth a thousand words: an analysis of the difficulty and discrimination parameters of illustrated vs. text-alone vignettes in histology multiple choice questions

Abstract

BackgroundAdvances in cognitive load theory have led to greater understanding of how we process verbal and visual material during learning, but the evidence base with regard to the use of images within written assessments is still sparse. This study examines whether the inclusion of images within the stimulus format of multiple choice questions (MCQs) has a predictable or consistent influence on psychometric item properties, such as difficulty or discrimination.MethodsItem analysis data from three consecutive years of histology multiple choice examinations were included in this study. All items were reviewed and categorised according to whether their stem, or stimulus format, was purely textual or included an associated image.ResultsA total of 195 MCQs were identified for inclusion and analysed using classical test theory; 95 used text alone and 100 included an image within the question stem. The number of students per examination ranged from 277 to 347, with a total of 60,850 student-question interactions. We initially examined whether the inclusion of an image within the item stem altered the item difficulty using Mann–Whitney U. The median item difficulty for images with purely textual stems was 0.77, while that for items incorporating an appropriate image was 0.80; this difference was not significant (0.77 vs. 0.80; p = 0.862, Mann–Whitney-U = 4818.5). Mean values showed that the Item Discrimination Index appeared unaffected by the inclusion of an image within the stem, and Item point biserial correlation also showed no difference in means between these two groups (Independent samples t-test; 2-tailed).ConclusionWe demonstrate that the addition of illustrations within undergraduate histology Multiple Choice Question stems has no overall influence on item difficulty, or measures of item discrimination. We conclude that the use of images in this context is statistically uncritical, and suggest that their inclusion within item stems should be based upon the principles of constructive alignment. However, further research with respect to the effect of images within item stems on cognitive processing, particularly with regard to image complexity or type, would enable the development of more informed guidelines for their use.

The effect of images on item statistics in multiple choice anatomy examinations

“…Finally, Holland et al () examined items in a histology course that utilized images and found no significant difference in difficulty or discrimination. Like the previously discussed study, they used images as part of the question stem that had to be interpreted to arrive at a correct answer.…”

“…More recently, Vorstenbosch et al () found increased item difficulty when using an image labeled format for students to select an answer to an item rather than selecting from a word list. Finally, Holland et al () found no difference in item difficulty or discrimination when using illustrated compared to text only vignettes in histology examinations. Other references to this line of inquiry are scarce, particularly in anatomical education although there does appear to be an increasing interest in this area.…”

Assessment of anatomical knowledge: Approaches taken by higher education institutions

“…Use of Images in Anatomy Assessment. Some studies have shown it makes no difference to test scores if images are included in the MCQ (Holland et al, 2015); others have found variable effects on item difficulty depending on the topic of the question (Vorstenbosch et al, 2013). Students' thought processes behind selecting the correct answer to MCQs with diagrams are described in Vorstenbosch et al (2014).…”

Effects of image‐based and text‐based active learning exercises on student examination performance in a musculoskeletal anatomy course

“…Of the 262 examination questions assessing material that was reinforced through active learning, a slight majority (60%) was associated with image‐based exercises. The percent correct and point serial correlation for each examination question were obtained from the university examination scoring service and were used to indicate examination item difficulty and discrimination (Holland et al, ).…”

“…Much of the previous work on visual and verbal formats have investigated the effects on learning in traditional, knowledge transmission settings (Butcher, ; Bartholomé and Bromme, ) or in assessments of learning (Hunt, ; Crisp and Sweiry, ; Berends and van Lieshout, ; Vorstenbosch et al, ; Holland et al, ). In this study, however, the effects of verbal and visual formats on learning were investigated in an active learning setting, in which students had time to think about their answers and discuss their answers with peers.…”