Evaluation Models for Criterion-Referenced Testing: Views Regarding Mastery and Standard-Setting

Abstract

In the dozen years since Glaser's (1963) seminal article on criterion-referenced testing, the acceptance of the concept of mastery as an educational and, hence, evaluation goal has grown tremendously. A large number of articles have been published, curriculum programs have been devised that employ criterionreferenced testing, and yet writers still feel it necessary to define what a criterion-referenced test is. Furthermore, the various published definitions are by no means equivalent. One also observes a shift in the interests and background of the authors of papers over this period. In the Sixties, writers were primarily advocating the adoption of criterion-referenced testing from an educational or philosophical point of view in spite of the reservations of the classical measurement theorists, whereas in the Seventies a new generation of measurement specialists have begun to be involved, and the papers are much more mathematical. A number of mathematically-based techniques for deciding

Proceedings of the Annual Conference of the Military Testing Association (19th), 17-21 October 1977, San Antonio, Texas

“…Either passing scores are arbitrarily set at some percent correct (Sanders, 1976;Shaycoft, 1976) or they are determined by complex mathematical methods that incorporate a and 0 classification errors (Emrick, 1971;Kriewall, 1972;Hively et al, 1973;Miliman, 1972 andRoundabush, 1974;). An a classification error occurs when a nonmaster is falsely deemed to be a master; conversely a 8 classification error occurs when a master is falsely deemed to be a nonmaster (Meskauskas, 1976). In most of the previous studies, the methods for determining passing scores were either too simplistic, thereby, resulting in large classification errors (Reichman and Oosterhoff, 1976, or requiring complex parameter estimation procedures (Wilcox and Harris, 1977).…”

“…This approach involves the use of Subject Matter Experts (SMEs) to define the minimal performance level for a test by rating the difficulty of each alternative to each test item for the minimally acceptable (just passina) examinee (Meskauskas, 1976). Thus, this procedure estabilshes item content rather than examinee perfornvance as ts.e basis for determining item difficulty (Smilansky and Guerin, 1976).…”

Binomial Test Models and Item Difficulty

“…According to this hypothesis, a student is not able to produce any correct response up to a certain point in the learning process; however, having passed this point, the situation has fully changed and he/she will always produce the correct answer. In the parlance appertaining to this hypothesis, knowledge is treated as an all-or-none state: A student who has passed the critical point in the learning process &dquo;knows&dquo; the item; the others &dquo;do not know it.&dquo; The deterministic view also underlies the so-called state models for mastery testing (see Besel, 1973;Dayton & Macready, 1976;Emrick, 1971;Emrick & Adams, 1969;Macready & Dayton, 1977;Meskauskas, 1976 Suppose that a stochastic conception of item responses (i.e., for a given person and item) is now adopted. Item responses are seen as the outcomes of a stochastic process dependent upon several person and item characteristics.…”

Criterion-Referenced Testing: A Critical Analysis of Selected Models

“…Meskauskas ( 1976) discusses several me thods that have been used to bridge the gap between operational tests and real-world requirements .…”