Preferred question types for computer-based assessment of clinical reasoning: a literature study

Abstract

Clinical reasoning is a core competence of doctors. Therefore, the assessment of clinical reasoning of undergraduate students is an important part of medical education. Three medical universities in the Netherlands wish to develop a shared question database in order to assess clinical reasoning of undergraduate students in Computer-Based Assessments (CBA). To determine suitable question types for this purpose a literature study was carried out. Search of ERIC and PubMed and subsequent cross referencing yielded 30 articles which met the inclusion criteria of a focus on question types suitable to assess clinical reasoning of medical students and providing recommendations for their use. Script Concordance Tests, Extended Matching Questions, Comprehensive Integrative Puzzles, Modified Essay Questions/Short Answer Questions, Long Menu Questions, Multiple Choice Questions, Multiple True/False Questions and Virtual Patients meet the above-mentioned criteria, but for different reasons not all types can be used easily in CBA. A combination of Comprehensive Integrative Puzzles and Extended Matching Questions seems to assess most aspects of clinical reasoning and these question types can be adapted for use in CBA. Regardless of the question type chosen, patient vignettes should be used as a standard stimulus format to assess clinical reasoning. Further research is necessary to ensure that the combination of these question types produces valid assessments and reliable test results.

Key-feature questions for assessment of clinical reasoning: a literature review

“…These questions are relatively easy to write and grade and can achieve reasonable reliability . However, because of time constraints, item writers tend to develop questions without sufficient detail, which, therefore, tend to measure only knowledge . The SCT format is based upon theory in cognitive psychology and upon script theory, and aims to measure clinical data interpretation in ill‐defined cases .…”

The Forgotten Dimension: The Information Content of Objective Questions

“…Objective questions have been used to test factual knowledge for many years. More recently, they have been used to test clinical reasoning (van Bruggen et al, 2012). This is a multi-step process using relevant facts in a logical sequence to reach a reasoned outcome.…”

“…The following agents may lead to the diagnosis you have chosen Choose the most likely principal diagnosis and causal agent for the clinical condition, Note: There were several clinical scenarios requiring responses to the same lists of possible answers Script Concordance Test (SCT) This is an example of a series of BOQs with a complex clinical stem and statement (van Bruggen, van Woudenbergh et al, 2012).…”

Long-menu questions in computer-based assessments: a retrospective observational study

“…CBA innovations include a new question format called “Long-menu questions”, which bear many positive features. This format assesses decision-making during diagnostic evaluation, diagnosis, and therapy [ 6 ]: the program narrows down the potential answers while the students type in their free text response, leaving the student with a number of options for their final selection. For that reason, Long-menu questions cannot be used in paper-based exams.…”