A COMPARISON OF FIVE DIFFERENT SCORING FUNCTIONS FOR CONFIDENCE TESTS<sup>1</sup>

Rippey, Robert M.

doi:10.1111/j.1745-3984.1970.tb00712.x

Cited by 30 publications

(16 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is striking that the greatest gain in reliability is reached using the nonreproducing, linear scoring rule. This finding is in line with results of Michael (1968) and Rippey (1970) . It means that the extra information the Brier scoring rule supplies about the way in which probabilities are assigned to all answers of the item introduces variance into the test scores not related to the ability level of the examinee.…”

Section: Discussionsupporting

confidence: 82%

Scoring rules and probability testing

Koele

Boo

Verschure

1987

Bull. Psychon. Soc.

View full text Add to dashboard Cite

When using multiple-choice items in educational testing, the examinee is usually forced to choose exactly one ofthe answer possibilities as the supposedly correct answer. An alternative response mode is to let examinees assign probabilities to each of the answers of the item, corresponding to their belief in the correctness of each answer. In this study student attitudes toward this response mode were investigated, and scoring rules for assessing item scores were compared. It appeared that students had no clear preference for either us ing subjective probabilities or making forced choices, and that from a psychometrical point of view scoring rules based only on the probability assigned to the correct answer have to be preferred to scoring rules using all probabilities assigned to each of the answers.In any form of psychological or educational testing, a distinction can be made between response mode and scoring rule . The response mode is the way in which the subject is required to respond to the stimulus material. The scoring rule prescribes how numerical values are assigned to the responses. When multiple-choice examinations are used to measure a specific ability , the common response mode is the forced choice; the examinee is requested to mark precisely one of the answer possibilities as the (supposedly) correct one. The usual scoring rule assigns an item score of one to the examinee if the correct answer has been marked and an item score of zero if not. The test score is then taken to be the sum of the item scores .The combination of this forced-choice response mode and dichotomous scoring rule has the advantage of a quick and efficient computerized processing of large numbers of examinations. On the other hand, there are two disadvantages. The forced-choice response mode introduces error variance into the test scores because of guessing, and the dichotomous scoring rule does not discriminate between different levels of partial knowledge. Both of these factors are a potential threat to the reliability of the test.One of the possible ways of overcoming these disadvantages is to use personal probabilities as a response mode to multiple-choice items. In its purest form, this response mode requires assigning a personal probability to each of the answers of a test item, according to the degree to which the answer is believed to be the correct one . Per item, these probabilities should add to unity. De Finetti (1965) gave an extensive overview of the different varieties of the response mode based on personal probabilities.The personal probability response mode requires a specific scoring rule , a function assigning a real number to an item based on the distribution of the personal probabilities over the answers of the item. Van Naerssen

show abstract

Section: Discussionsupporting

confidence: 82%

Scoring rules and probability testing

Koele

Boo

Verschure

1987

Bull. Psychon. Soc.

View full text Add to dashboard Cite

show abstract

“…Rippey (1970) compared five such approaches and summarized the re sults as follows: "Comparison between reliabilities showed that the simplest and most intui tive function, the probability assigned to the correct answer, produced the highest reliabil ity.. . " Of the five funtions evaluated, the probability that a candidate assigns to the cor rect answer most closely approximates the self-weighting functions described in this paper.…”

Section: Discussionmentioning

confidence: 99%

Increasing Testing Efficiency and Effectiveness per Item and per Minute of Testing Time

Jensen¹

1983

Public Personnel Management

View full text Add to dashboard Cite

“…It may increase the reliability (Rippey, 1970). If the test is valid initially, it will still be valid using this probability format.…”

Section: Advantagesmentioning

confidence: 96%

“…The aim of this work, however, was to obtain insight into student conceptualisations, rather than to award marks for partial understanding. Similar proposals, but with an emphasis on scoring procedures, have been put forward since at least the 1960s in education (e.g., Michael, 1968;Rippey, 1968Rippey, , 1970; these are reviewed in Hutchinson, 1991, pp. 84 -93) and earlier still in psychophysical work (e.g., Hollingworth, 1913).…”

mentioning

confidence: 94%

See 1 more Smart Citation

An alternative method of answering and scoring multiple choice tests

Taylor¹,

Gardner

1999

Research in Science Education

View full text Add to dashboard Cite

A simple modification to the method of answering and scoring multiple choice tests allows students to indicate their estimates of the probability of the correcmess of the multiple choice options for each question, without affecting the validity of the assessment. A study was conducted using a test that investigated common misconceptions in mechanics. The study showed that for assessment purposes this method gives results that are very similar to results obtained by students who answer in the traditional manner. Year 12 Physics students (N = 85) were randomly allocated to two treatment groups: one received a standard format multiple choice test, the other a test format allowing students to select more than one response in a multiple choice test, and to distribute their marks among their chosen options. An analysis of the students' uncertainties is used to argue that not only can students appeal to different conceptions in different contexts, but that they can also hold conflicting conceptions with respect to a single context.

show abstract

A COMPARISON OF FIVE DIFFERENT SCORING FUNCTIONS FOR CONFIDENCE TESTS¹

Cited by 30 publications

References 3 publications

Scoring rules and probability testing

Scoring rules and probability testing

Increasing Testing Efficiency and Effectiveness per Item and per Minute of Testing Time

An alternative method of answering and scoring multiple choice tests

Contact Info

Product

Resources

About

A COMPARISON OF FIVE DIFFERENT SCORING FUNCTIONS FOR CONFIDENCE TESTS1

Cited by 30 publications

References 3 publications

Scoring rules and probability testing

Scoring rules and probability testing

Increasing Testing Efficiency and Effectiveness per Item and per Minute of Testing Time

An alternative method of answering and scoring multiple choice tests

Contact Info

Product

Resources

About

A COMPARISON OF FIVE DIFFERENT SCORING FUNCTIONS FOR CONFIDENCE TESTS¹