Polytomous versus Dichotomous Scoring on Multiple-Choice Examinations: Development of a Rubric for Rating Partial Credit

Abstract

The concept of assigning partial credit on multiple-choice test items is considered for items from ACS Exams. Because the items on these exams, particularly the quantitative items, use common student errors to define incorrect answers, it is possible to assign partial credits to some of these incorrect responses. To do so, however, it becomes vital that instructors reach general agreement as to the level of partial credit. Using workshops with instructors, ACS Exams has identified reasons why partial credit could be assigned in an exam set of 70 test items. With partial-credit assignments thus established, polytomous scoring is applied and the effect of such scoring on the overall norm-referenced psychometrics is determined and described. While individual students move within the overall norm, the average influence of polytomous scoring as conceived by the workshops does not substantially change the ability to do norm-based comparisons. ABSTRACT: The concept of assigning partial credit on multiple-choice test items is considered for items from ACS Exams. Because the items on these exams, particularly the quantitative items, use common student errors to define incorrect answers, it is possible to assign partial credits to some of these incorrect responses. To do so, however, it becomes vital that instructors reach general agreement as to the level of partial credit. Using workshops with instructors, ACS Exams has identified reasons why partial credit could be assigned in an exam set of 70 test items. With partial-credit assignments thus established, polytomous scoring is applied and the effect of such scoring on the overall norm-referenced psychometrics is determined and described. While individual students move within the overall norm, the average influence of polytomous scoring as conceived by the workshops does not substantially change the ability to do norm-based comparisons.

The Development of HOTS Test Instrument of Thermodynamics Law for Senior High School

“…Each category has different difficulty levels for each item. This result agrees with the research of Grunert et al (2013), which states that the use of partial credit, which is divided into several categories gives a significant impact on the item being tested. The results of the research in Table 6 on the difficulty column show each item's difficulty level.…”

“…Table 6 shows a summary of parameter estimates analyzed using the PARSCALE 4 program. Grunert et al (2013), which states that the use of partial credit which is divided into several categories. Each category has different difficulty levels for each item.…”

Assessing Database Development Skills using an On-line MCQ: Reflections on Test Design and Academic Integrity

“…On a well-designed MCQ test, the correct answer may not be too far removed from very similar incorrect answers. Based on this rationale, it was decided not to use negative marking for incorrect answers, and also to award partial marks for questions where there was a statistically skewed distribution of incorrect responses, similar to the approach suggested by Grunert et al (2013).…”

Analysing the distribution of questions in the gas law chapters of secondary and introductory college chemistry textbooks from the United States

“…In a subsequent paper, Johnstone and Ambusaidi (2001) described other forms of fixed-response questions that attempt to address the disadvantages of traditional multiple-choice questions mentioned in their previous work. Recently, the Examinations Institute of the American Chemical Society has reported an effort to provide students with partial credit on their examinations instead of simply scoring the question right or wrong (Grunert et al, 2013). They found that students on either extreme did not see much difference in their scores with or without partial credit; however, the middleperforming students showed the most potential for change but these changes were evenly split (half showing increased scores while the other half showing decreased scores).…”