MATCHED PAIR TRUE‐FALSE SCORING: EFFECT ON RELIABILITY AND VALIDITY¹

Abstract

The matched pair technique for writing and scoring true-false items was designed to compensate for the acquiescence response set of primary grade children. The claim that this technique increases reliability to an appreciable extent over traditional true-false scoring was investigated by comparing alpha internal consistency coefficients computed for the matched pair true-false, traditional true-false, and three other scoring schemes. Both the total sample coefficients and individual classroom coefficients were computed from the standardization sample of a primary grade economics achievement test (Primary Test of Economic Understanding). Classroom reliability coetScients computed from the matched pair scores were found to be higher than those from scores computed by the other methods. Total sample coefficients obtained from four of the five methods were nearly equal. Evidence of the effects of each scoring technique on concurrent validity is also presented. Contrary to expectations, the correlations of traditional and matched pair scores with Iowa Test of Basic Skills (ITBS) subtests (when adjusted for differing reliabilities) were approximately equal.

Multiple True‐False Items: A Study of Interitem Correlations, Scoring Alternatives, and Reliability Estimation

“…Differences in test difficulty may in part account for the conflicting results obtained in the present study when compared with those obtained by Brandenburg and Whitney (1972) in their study of paired true-false items. In that study, the mean percentage correct was 68% as opposed to 87% correct in the present study.…”

“…The desire to reduce the probability of chance success has led to suggestions for longer tests and more options per multiple-choice item (Ebel, 1969;Wesman, 1971). The MR scoring method has a counterpart in matchedpair true-false scoring introduced by Larkins and Shaver (1968) and studied by Brandenburg and Whitney (1972).…”