A preliminary investigation of three types of multiple choice questions

Skakun, Ernest N.; Nanson, Eric M.; Kling, Sarah M.; Taylor, William C.

doi:10.1111/j.1365-2923.1979.tb00928.x

Cited by 24 publications

(11 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although Skakun et al (1979) reported that A‐, K, and E‐ (assertion‐reason) type items were unable to discriminate among cognitive traits, our department has, over the years, used three different item‐types in anatomy MCQ tests. According to Lipton & Huxham (1974), an important need met by such a practice is defined by the fact that the presentation of certain problems is often better suited to a given item‐type than to others.…”

Section: Discussionmentioning

confidence: 99%

Multiple-choice testing in anatomy

Nnodim

1992

Medical Education

View full text Add to dashboard Cite

An analysis of 596 multiple-choice questions (MCQs) on human anatomy given at three First Professional Examinations for medical students is reported. The MCQ paper at each examination was 200 items long and consisted of three item-types: A, K and T/F. Each A-type item comprised a stem and five options, only one of the latter being the correct or best answer. Items of the K-type consisted of a stem and four responses, any number of which may be correct. The T/F items were of the three-response kind, the available options being 'true', 'false' and 'don't know'. Test reliability was computed by internal analysis, using the Kuder-Richardson 20 formula. Measures of concurrent validity were obtained by correlating the scores in the MCQ papers with the overall outcome of the First Professional Examination. Indices of item facility, discrimination and abstention were calculated. The effects of item-type and the availability of the 'don't know' option on examinee performance were also determined. Reliability (alpha) and concurrent validity (Pearson r) coefficients in the ranges of 0.71-0.85 and 0.80-0.93 (P less than 0.05) respectively were recorded. Regression analysis revealed the MCQ papers to be less sensitive predictors of the aggregate performance than the essay papers. The proportion of highly discriminatory and excessively difficult items was highest for the K-type. When the same K-type questions were re-exhibited in the indeterminate format, the examinees performed significantly better. Higher scores were also recorded when candidates were required to respond to all the questions than when they were offered the 'don't know' option and the percentage gain was higher for the low-scoring examinees. The appropriateness of multiple-choice testing as a tool for assessing student achievement in human anatomy is discussed.

show abstract

Section: Discussionmentioning

confidence: 99%

Multiple-choice testing in anatomy

Nnodim

1992

Medical Education

View full text Add to dashboard Cite

show abstract

“…It is quite surprising, therefore, that the academic literature on the subject is quite sparse. Connelly (2004) provides an overview of the existing published material (see Newble et al, 1979;Skakun et al, 1979;Fox, 1983;and Scouller & Prosser, 1994) before going on to extol the main virtue of the ARQ test item; viz. that 'its structure facilitates the construction of questions that test student learning beyond recall.…”

Section: Assertion-reason Multiple-choice Testing 291mentioning

confidence: 99%

Assertion‐reason multiple‐choice testing as a tool for deep learning: a qualitative analysis

Williams

2006

Assessment & Evaluation in Higher Education

View full text Add to dashboard Cite

This paper reflects on the ongoing debate surrounding the usefulness (or otherwise) of multiplechoice questions (MCQ) as an assessment instrument. The context is a graduate school of business in Australia where an experiment was conducted to investigate the use of assertion-reason questions (ARQ), a sophisticated form of MCQ that aims to encourage higher-order thinking on the part of the student. It builds on the work of Connelly (2004) which produced a quantitative analysis of the use of ARQ testing in two economics course units in a flexibly-delivered Master of Business Administration (MBA) program. Connelly's main findings were that ARQ tests were good substitutes for the more conventional type of multiple-choice/short-answer type questions and, perhaps more significantly, ARQ test performance was a good predictor of student performance in essays-the assessment instrument most widely favoured as an indicator of deeper learning. The main focus of this paper is the validity of the second of these findings, analysis of questionnaire data casting some doubt over whether student performance in ARQ tests can, indeed, be looked upon as a sound indicator of deeper learning-student reactions and opinions suggesting instead that performance might have more to do with one's proficiency in the English language.

show abstract

“…2,3 In recent years, the most common type of MCQs employed in examinations are type A MCQs, which consist of a stem followed by four or five options or distractors. 4,5 An item analysis assesses the reliability and validity of an examination by examining student performance with regards to each MCQ and applying statistical analyses to determine whether the item should be kept, reviewed or discarded from the test. Common item analysis parameters include the difficulty index (DIFI), which reflects the percentage of correct answers to total responses; the discrimination index (DI), also known as the point biserial correlation, which identifies discrimination between students with different levels of achievement; and distractor efficiency (DE), which indicates whether the distractors in the item are well-chosen or have failed to distract students from selecting the correct answer.…”

mentioning

confidence: 99%

Item Analysis of Multiple Choice Questions at the Department of Paediatrics, Arabian Gulf University, Manama, Bahrain

Kheyami

Jaradat

Al-Shibani

et al. 2018

Sultan Qaboos Univ Med J

View full text Add to dashboard Cite

Objectives: The current study aimed to carry out a post-validation item analysis of multiple choice questions (MCQs) in medical examinations in order to evaluate correlations between item difficulty, item discrimination and distraction effectiveness so as to determine whether questions should be included, modified or discarded. In addition, the optimal number of options per MCQ was analysed. Methods: This cross-sectional study was performed in the Department of Paediatrics, Arabian Gulf University, Manama, Bahrain. A total of 800 MCQs and 4,000 distractors were analysed between November 2013 and June 2016. Results: The mean difficulty index ranged from 36.70-73.14%. The mean discrimination index ranged from 0.20-0.34. The mean distractor efficiency ranged from 66.50-90.00%. Of the items, 48.4%, 35.3%, 11.4%, 3.9% and 1.1% had zero, one, two, three and four nonfunctional distractors (NFDs), respectively. Using three or four rather than five options in each MCQ resulted in 95% or 83.6% of items having zero NFDs, respectively. The distractor efficiency was 91.87%, 85.83% and 64.13% for difficult, acceptable and easy items, respectively (P <0.005). Distractor efficiency was 83.33%, 83.24% and 77.56% for items with excellent, acceptable and poor discrimination, respectively (P <0.005). The average KuderRichardson formula 20 reliability coefficient was 0.76. Conclusion: A considerable number of the MCQ items were within acceptable ranges. However, some items needed to be discarded or revised. Using three or four rather than five options in MCQs is recommended to reduce the number of NFDs and improve the overall quality of the examination. Keywords

show abstract

A preliminary investigation of three types of multiple choice questions

Cited by 24 publications

References 3 publications

Multiple-choice testing in anatomy

Multiple-choice testing in anatomy

Assertion‐reason multiple‐choice testing as a tool for deep learning: a qualitative analysis

Item Analysis of Multiple Choice Questions at the Department of Paediatrics, Arabian Gulf University, Manama, Bahrain

Contact Info

Product

Resources

About