The Use of Confidence Testing In Objective Tests

Abstract

Lord and Novick (1968) have stated that the general problem of obtaining the maximum amount of information from a given set of items contains three major components. The first of these is the measurement procedure, or the manner in which the examinees respond to the items. The second component is the specification of an item scoring rule or formula that is used for each item. The final component is the combination of the item scores into a total score by an item weighting formula. The first two components comprise much of the subject of confidence testing. Confidence testing is a method of testing where weights are assigned directly or indirectly to item responses in such a way as to reflect the examinee's belief in the correctness of the alternative or alternatives so marked. One author (Jacobs, 1971) has made a distinction between confidence weighting procedures and probabilistic testing. This paper considers these two procedures as categories of the more general subject of confidence testing.The purpose of this paper is to describe the various forms of confidence testing as they have been developed and to provide a brief evaluation of these forms. Some of the problems associated with confidence testing are discussed with the relevant studies cited.In the usual multiple-choice test, an examinee is given a question along with a number of possible answers to that question. He is then asked to choose an answer from those given and to indicate that choice on an answer sheet. If he chooses the correct answer, he receives a score of one; if he omits the item, he gets a score of zero; and if he guesses incorrectly or is misinformed as to the correct answer, he receives a score less than or equal to zero, as with the commonly used formula score. The total test score is then taken as the sum of the item scores.This type of testing has the advantages of efficiency and simplicity for both the examinee and the test scorer. More items can be administered in a given period of time using this method than by any other method requiring a more complicated response, and the cost for scoring the test is also less.Advocates of confidence testing have stated that knowledge is neither a dichotomous nor a trichotomous affair, which conventional 217 at UNIV CALIFORNIA SAN DIEGO on June 10, 2015 http://rer.aera.net Downloaded from

Exploring Pre-Service Teachers’ Conceptual Understanding and Confidence in Geometrical Optics: A Focus on Gender and Prior Course Achievement

“…Since the 1970s, psychology has extensively investigated the relationship between performance and confidence [33,49]. Up to now, however, few studies have been applied in physics [19,20,[23][24][25]27,28,[50][51][52].…”

Belief Elicitation: A Horse Race among Truth Serums

“…Beliefs can be measured through scoring rules, introduced by Brier () and de Finetti (), which have the advantage that a single choice suffices to determine the exact belief in an incentive compatible way. Scoring rules have been used in many domains, including education (Echternacht, ), finance (Shiller et al ., ), political science (Tetlock, ) and experimental game theory (Costa‐Gomes and Weizsäcker, ). The most popular scoring rule is the QSR (McKelvey and Page, ; Nyarko and Schotter, ).…”