eWorkbook

Costagliola, Gennaro; Ferrucci, Filomena; Fuccella, Vittorio; Oliveto, Rocco

doi:10.4018/jdet.2007070103

Cited by 8 publications

(2 citation statements)

References 16 publications

(16 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For incorrect responses, 1 – 1/n credit points are awarded. The formula is S = R + (1 – 1/n)W. This scoring method was first described by Costagliola et al [ 154 ] in 2007 and named fair penalty by the authors. However, the term penalty is misleading because no points are deducted in case of incorrect responses.…”

Section: Resultsmentioning

confidence: 99%

Scoring Single-Response Multiple-Choice Items: Scoping Review and Comparison of Different Scoring Methods

Kanzow¹,

Schmidt²,

Kanzow³

2023

JMIR Med Educ

View full text Add to dashboard Cite

Background Single-choice items (eg, best-answer items, alternate-choice items, single true-false items) are 1 type of multiple-choice items and have been used in examinations for over 100 years. At the end of every examination, the examinees’ responses have to be analyzed and scored to derive information about examinees’ true knowledge. Objective The aim of this paper is to compile scoring methods for individual single-choice items described in the literature. Furthermore, the metric expected chance score and the relation between examinees’ true knowledge and expected scoring results (averaged percentage score) are analyzed. Besides, implications for potential pass marks to be used in examinations to test examinees for a predefined level of true knowledge are derived. Methods Scoring methods for individual single-choice items were extracted from various databases (ERIC, PsycInfo, Embase via Ovid, MEDLINE via PubMed) in September 2020. Eligible sources reported on scoring methods for individual single-choice items in written examinations including but not limited to medical education. Separately for items with n=2 answer options (eg, alternate-choice items, single true-false items) and best-answer items with n=5 answer options (eg, Type A items) and for each identified scoring method, the metric expected chance score and the expected scoring results as a function of examinees’ true knowledge using fictitious examinations with 100 single-choice items were calculated. Results A total of 21 different scoring methods were identified from the 258 included sources, with varying consideration of correctly marked, omitted, and incorrectly marked items. Resulting credit varied between –3 and +1 credit points per item. For items with n=2 answer options, expected chance scores from random guessing ranged between –1 and +0.75 credit points. For items with n=5 answer options, expected chance scores ranged between –2.2 and +0.84 credit points. All scoring methods showed a linear relation between examinees’ true knowledge and the expected scoring results. Depending on the scoring method used, examination results differed considerably: Expected scoring results from examinees with 50% true knowledge ranged between 0.0% (95% CI 0% to 0%) and 87.5% (95% CI 81.0% to 94.0%) for items with n=2 and between –60.0% (95% CI –60% to –60%) and 92.0% (95% CI 86.7% to 97.3%) for items with n=5. Conclusions In examinations with single-choice items, the scoring result is not always equivalent to examinees’ true knowledge. When interpreting examination scores and setting pass marks, the number of answer options per item must usually be taken into account in addition to the scoring method used.

show abstract

Section: Resultsmentioning

confidence: 99%

Scoring Single-Response Multiple-Choice Items: Scoping Review and Comparison of Different Scoring Methods

Kanzow¹,

Schmidt²,

Kanzow³

2023

JMIR Med Educ

View full text Add to dashboard Cite

show abstract

“…The system has been given a Web-based interface. Rather than developing it from scratch, we have preferred to integrate the system in an existing Web-based e-testing platform: eWorkbook (Costagliola et al;2007), developed at the University of Salerno.…”

Section: Introductionmentioning

confidence: 99%

A Rule-Based System for Test Quality Improvement

Costagliola

Fuccella

2009

International Journal of Distance Education Technologies

Self Cite

View full text Add to dashboard Cite

To correctly evaluate learners' knowledge, it is important to administer tests composed of good quality question items. By the term "quality" we intend the potential of an item in effectively discriminating between skilled and untrained students and in obtaining tutor's desired difficulty level. This paper presents a rulebased e-testing system which assists tutors in obtaining better question items through subsequent test sessions. After each test session, the system automatically detects items' quality and provides the tutors with advice about what to do with each of them: good items can be re-used for future tests; among items with lower performances, instead, some should be discarded, while some can be modified and then re-used. The proposed system has been experimented in a course at the University of Salerno.

show abstract