Effect of Perceived Scoring Formula On Some Aspects of Test Performance

Waters, L. K.

doi:10.1177/001316446702700448

Cited by 7 publications

(5 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is, to the author's knowledge, no direct empirical evidence for this suggestion where CTD are concerned. However, if one extends the argument "by analogy" from the similar results obtained in the Sherriffs and Boomer (1954) and Hritz and Jacobs (1970) studies, the results of Waters (1967) may be relevant. Waters found that increased levels of penalty resulted in significant increases in the number of omitted items in a conventional multiple-choice testing situation.…”

mentioning

confidence: 75%

Behavior on Objective Tests Under Theoretically Adequate, Inadequate and Unspecified Scoring Rules

Jacobs

1975

J Educational Measurement

View full text Add to dashboard Cite

A number of alternative administrative and scoring procedures for objective tests have been suggested (e.g., Coombs, 1953;de Finetti, 1965;Ebel, 1965;Rippey, 1968) which have as their common objective a more adequate assessment of the degree of partial knowledge held by a given student with reference to a given item?A procedure known as 'option-elimination' or "Coombs-type-directions' (CTD) seems quite applicable to the typical classroom testing situation. With CTD, the student is required to identify as many of the J -1 distractors among the J item options as he or she is able. With the usual scoring rule, a student earns one point for each distractor so identified. A penalty of -(J -1) points is suffered if the correct answer is identified as a distractor. Item scores, then, can range from -(J -1) points to +(J -1) points, having 2(J -1) + 1 possible values, rather than simply the 1 or 0 earned under conventional conditions. Apparently, CTD have the potential for discerning intermediate levels of knowledge.Hritz and Jacobs (1970) demonstrated that under CTD the problems associated with correction-for-guessing (Sherriffs and Boomer, 1954;Votaw, 1936) have apparently been simply shifted from the item level to the option level. In a study which used a four-option multiple choice vocabulary test as the research vehicle, Hritz and Jacobs (1970) found that when students were stratified on the basis of "nonsense items" and the use of Slakter's (1967) formula (essentially the proportion of the options of the nonsense items responded to by S), there were marked differences among the strata in the number of options eliminated on legitimate items (i.e., not identified as distractors). That is, the behavior noted on the nonsense items was highly predictive of behavior on the legitimate items, which is reasonable, since Ss were not privy to the distinction. However, it was found that when Ss were required to answer every item in a conventional fashion (by identifying the correct answer) the proportions of successful identifications were not significantly different across strata. This implies that the ability to identify distractors should not vary significantly among the strata. In addition to what appear to be significantly different tendencies in their readiness to identify some number of the k(J-1) distractors on a k-item test, Hritz and Jacobs (1970) found all Ss to be too conservative--all Ss identified too few distractors despite the fact that, when required, they could identify the correct answer with a frequency greater than that predicted by their CTD performance. There seem to be reliable and extreme individual differences in the tendency to respond to items under an announced guessing penalty

show abstract

mentioning

confidence: 75%

Behavior on Objective Tests Under Theoretically Adequate, Inadequate and Unspecified Scoring Rules

Jacobs

1975

J Educational Measurement

View full text Add to dashboard Cite

show abstract

“…The failure of examinee behavior to conform to the knowledge-or-random-guessing model has been observed and reported for decades. In some of this research, the observation that examinees both guess and make mistakes is related to the study's purpose (e.g., Ebel, 1968;Foster & Ruch, 1927;Little & Creaser, 1968;Melican, Mills, & Plake, 1989;Rowley & Traub, 1977;Sanderson, 1973;Waters, 1967;Wood, 1926). In other cases, this observation is merely incidental (e.g., Bridgeman, 1992;Campbell, 2015;Holzinger, 1924;Jackson, 1955;Michael, Stewart, Douglass, & Rainwater, 1963;Ruch & DeGraff, 1926;Sherriffs, & Boomer, 1954;Slakter, 1968;Swineford & Miller, 1953;Traub, Hambleton, & Singh, 1969;Votow, 1936;Wood, 1976).…”

Section: Knowledge or Guessingmentioning

confidence: 99%

A Problem with the Bookmark Procedure's Correction for Guessing

Baldwin

2020

Educational Measurement

View full text Add to dashboard Cite

In the Bookmark standard-setting procedure, panelists are instructed to consider what examinees know rather than what they might attain by guessing; however, because examinees sometimes do guess, the procedure includes a correction for guessing. Like other corrections for guessing, the Bookmark's correction assumes that examinees either know the correct answer or guess. It is widely known that this assumption is not typically met in practice and while there may be applications that are robust to its violation, the Bookmark procedure is not one of them. Analytical results show that when this assumption is not met, the Bookmark procedure overcorrects and expected cut score estimates will be too high (stringent).

show abstract

“…It is frequently found that a higher-than-chance proportion of these reluctantly answered items are in fact correct. Thus if either number-right scoring or the conventional guessing correction are used, the subjects increase their scores when forced to respond to items not initially answered (Votaw, 1936;Sherriffs & Boomer, 1954;Waters, 1967;Slakter, 1968a;Pyrczak, 1977;Bliss, 1980). In some of these studies, independent measures of personality were obtained, and it was sometimes found that this affects the degree to which subjects' scores are reduced by reluctance to answer questions they are unsure about, the more submissive or anxious type of personality being the most penalized.…”

Section: Reluctant Responsementioning

confidence: 99%

“…Whether the same subjects are taking similar tests (Hammerton, 1965), or similar subjects are taking the same test (Slakter, 1968b), higher scores are obtained in the latter condition. In addition, Waters (1967) found that the fraction of unanswered items t h a t are correct when an answer is forced is higher in groups which have been most deterred (by knowledge of the scoring formula to be used) from answering.…”

Section: Manipulation Of Instructionsmentioning

confidence: 99%

Some theories of performance in multiple choice tests, and their implications for variants of the task

Hutchinson¹

1982

Brit J Math & Statis

View full text Add to dashboard Cite

Theories are put forward that attempt to answer the practical question of 'How should we correct for guessing in multiple choice tests?' and the theoretical question of 'How can we mathematically describe partial knowledge so as to predict behaviour in tasks which enable it to be shown?'. Empirical findings relating to performance on variants of the multiple choice task are reviewed, and compared to the predictions of the theories.

show abstract

Effect of Perceived Scoring Formula On Some Aspects of Test Performance

Cited by 7 publications

References 4 publications

Behavior on Objective Tests Under Theoretically Adequate, Inadequate and Unspecified Scoring Rules

Behavior on Objective Tests Under Theoretically Adequate, Inadequate and Unspecified Scoring Rules

A Problem with the Bookmark Procedure's Correction for Guessing

Some theories of performance in multiple choice tests, and their implications for variants of the task

Contact Info

Product

Resources

About