Study on item and test analysis of multiple choice questions amongst undergraduate medical students

Abstract

INTRODUCTIONItem analysis is the process of collecting, summarizing and using information from students' response to assess the quality of test items.1,2 However it is said that MCQs emphasize recall of factual information rather than conceptual understanding and interpretation of concepts. 3There is more to writing good MCQs than writing good questions. Properly constructed MCQs can assess higher cognitive processing of Bloom's taxonomy such as interpretation, synthesis and application of knowledge, instead of just testing recall of isolated facts. 4,5 A multiple-choice question (MCQ) consists of a stem with a question line at its end or underneath it, followed by a number of options. One of the options is the correct or best response known as the key, while the others are described as distracters. 6 An essential characteristic of distracters is that all options shall present plausible answers and if possible none shall be incorrect.7 Function of a distracter is to attract students who do not know the correct answer while students who know the correct answer ignore them. Multiple choice questions (MCQ'S) are used as an objective and reliable tool to evaluate learning performance of students. It is also a preferred tool for selection of students for a given course. MCQ needs to be tested for the standard or quality.8 However the items to be used must be of good quality to serve the purpose for it is meant to be. Objectives To assess the item and test quality of multiple choice questions. ABSTRACTBackground: Item analysis is the process of collecting, summarizing and using information from students' response to assess the quality of test items. However it is said that MCQs emphasize recall of factual information rather than conceptual understanding and interpretation of concepts. There is more to writing good MCQs than writing good questions. The objectives of the study was to assess the item and test quality of multiple choice questions and to deal with the learning difficulties of students, identify the low achievers in the test. Methods:The hundred MBBS students from Government medical college were examined. A test comprising of thirty MCQs was administered. All items were analysed for Difficulty Index, Discrimination Index and Distractor Efficiency. Data entered in MS Excel 2007 and SPSS 21 analysed with statistical test of significance. Results: Majority 80% items difficulty index is within acceptable range. 63% items showed excellent discrimination Index. Distractor efficiency was overall satisfactory. Conclusions: Multiple choice questions with average difficulty and also having high discriminating power with good distracter efficiency should be incorporated into student's examination.

Competitive Learning Using a Three-Parameter Logistic Model

“…This research is the first study to use IRT to look at the application of a multiple-choice test in a university-level statistics competition that promotes active and competitive learning among students. However, a literature review finds similar studies using IRT for partial and departmental exams from a wide range of undergraduate courses (Awopeju, Afolabi, 2016;Balmori et al, 2011;DiBattista, Kurzawa, 2011;Escudero et al, 2000;Gajjar et al, 2014;Ingale et al, 2017;Marie, Edannur, 2015;Mitra et al, 2009;Rao et al, 2016;Romero et al, 2015).…”

“…Among the extensive literature that considers the application of tests of abilities and knowledge in university settings, the so-called Item Response Theory (IRT) can be found. This theory, among other things, serves to analyze curricular content, adequate item design, the recognition of teaching-learning problems and the identification of students with low or high academic ability (Awopeju, Afolabi, 2016;Balmori et al, 2011;DiBattista, Kurzawa, 2011;Ingale et al, 2017;Mitra et al, 2009;Rao et al, 2016;Romero et al, 2015).…”

Item Analysis of Multiple Choice Questions in Medical Licensing Examination

“…The proportion of students answering an item correctly indicates the level of difficulty of the item, and it is affected by the examinees' knowledge, as well as the clarity and relevance of the question. Practically, it is essential to use items with positive discrimination level when implementing MCQ in medical licensing examination [10][11][12][13][14][15][16] .…”

Analysis of Competitive Learning at University Level in Mexico via Item Response Theory

“…Various studies have analyzed knowledge examinations at a university level using Item Response Theory (IRT) to evaluate course topics, analyzing both the quality of the items and difficulties with learning, as well as identifying badly performing students (Awopeju & Afolabi, 2016;Balmori, Delgadillo, & Méndez, 2011;DiBattista & Kurzawa, 2011;Ingale, Giri, & Doibale, 2017;Mitra, Nagaraja, Ponnudurai, & Judson, 2009;Rao, Kishan Prasad, Sajitha, Permi, & Shetty, 2016;Romero, Rojas, Domínguez, Pérez, & Sapsin, 2015).…”

“…Of the studies on partial knowledge tests or those graduating from undergraduate degree programs, various have used IRT (Awopeju & Afolabi, 2016;Balmori et al, 2011;DiBattista & Kurzawa, 2011;Escudero, Reyna, & Morales, 2000;Gajjar, Sharma, Kumar, & Rana, 2014;Ingale et al, 2017;Marie & Edannur, 2015;Mitra et al, 2009;Rao et al, 2016;Romero et al, 2015). However, from the perspective of the authors of this study, this is the first paper to analyze a test taken from a statistics knowledge competition using IRT, by means of which it sought to identify university students' competitive learning.…”