Coefficients for Tests from a Decision Theoretic Point of View

Linden, Willem J. van der; Mellenbergh, Gideon J.

doi:10.1177/014662167800200112

Cited by 17 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the psychometric literature, several methods for computing and interpreting reliability indexes for decisions have been proposed (Hambleton & Novick, 1973;Huynh, 1976b;Subkoviak, 1976;Swaminathan, Hambleton, & Algina, 1974;van der Linden & Mellenbergh, 1978). Two reliability indexes that have been recommended are (a) the agreement coefficient and (b) the kappa coefficient.…”

Section: A Numerical Examplementioning

confidence: 99%

See 1 more Smart Citation

Simultaneous Optimization of Decisions Using a Linear Utility Function

Vos

1990

Journal of Educational Statistics

View full text Add to dashboard Cite

The purpose of this article is to simultaneously optimize decision rules for combinations of elementary decisions. With this approach, rules are found that make more efficient use of the data than could be achieved by optimizing these decisions separately. The framework for the approach is derived from Bayesian decision theory. To illustrate the approach, two elementary decisions (selection and mastery decisions) are combined into a simple decision network. A linear utility structure is assumed. Decision rules are derived both for quota-free and quota-restricted selection–mastery decisions in case of several subpopulations. An empirical example of instructional decision making in an individual study system concludes the article.

show abstract

Section: A Numerical Examplementioning

confidence: 99%

“…An elegant typology of test-based decisions has been given in van der Linden (1985van der Linden ( , 1990. Each type of decision making in this typology can be viewed as a specific configuration of three basic elementsnamely, a test, a treatment, and a criterion.…”

mentioning

confidence: 99%

Simultaneous Optimization of Decisions Using a Linear Utility Function

Vos

1990

Journal of Educational Statistics

View full text Add to dashboard Cite

show abstract

“…For deriving optimal decision procedures, it is necessary to use a loss function; and in order to assess the optimality of the resulting decision procedure, a coefficient is needed that is based on the loss function that is actually used. Coefficient delta (van der Linden & Mellenbergh, 1978) has this property. The coefficient is defined for a broad class of decision situations and loss functions; results for some special cases are known.…”

mentioning

confidence: 93%

“…The external optimality is the risk of the decision procedure with respect to an external criterion. For assessing the optimality of a decision procedure, coefficient delta (van der Linden & Mellenbergh, 1978), which can be considered a standardization of the Bayes risk or expected loss, can be used. Two loss functions are dealt with: the threshold and the linear loss functions.…”

mentioning

confidence: 99%

See 1 more Smart Citation

The Internal and External Optimality of Decisions Based on Tests

Mellenbergh

Linden

1979

Applied Psychological Measurement

Self Cite

View full text Add to dashboard Cite

In applied measurement, test scores are usually transformed to decisions. Analogous to classical test theory, the reliability of decisions has been defined as the consistency of decisions on a test and a retest or on two parallel tests. Coefficient kappa (Cohen, 1960) is used for assessing the consistency of decisions. This coefficient has been developed for assessing agreement between nominal scales. It is argued that the coefficient is not suited for assessing consistency of decisions. Moreover, it is argued that the concept consistency of decisions is not appropriate for assessing the quality of a decision procedure. It is proposed that the concept consistency of decisions be replaced by the concept optimality of the decision procedure. Two types of optimality are distinguished. The internal optimality is the risk of the decision procedure with respect to the true score the test is measuring. The external optimality is the risk of the decision procedure with respect to an external criterion. For assessing the optimality of a decision procedure, coefficient delta (van der Linden & Mellenbergh, 1978), which can be considered a standardization of the Bayes risk or expected loss, can be used. Two loss functions are dealt with: the threshold and the linear loss functions. Assuming psychometric theory, coefficient delta for internal optimality can be computed from empirical data for both the threshold and the linear loss functions. The computation of coefficient delta for external optimality needs no assumption of psychometric theory. For six tests coefficient delta as an index for internal optimality is computed for both loss functions; the results are compared with coefficient kappa for assessing the consistency of decisions with the same tests.

show abstract

Obituary GIDEON J. MELLENBERGH (1938–2021)

2021

Self Cite

View full text Add to dashboard Cite

presided accreditation committees to evaluate Dutch and Flemish university programs. Don was a connecting figure who was able to build and maintain good relationships with all members of the Dutch psychometric community, even where there were passionate differences of opinion about psychometrics. He did not care for schools of thought; he only distinguished between good and bad research-and disliked the latter. This motivated a lifelong quest to improve research methods in psychology.Don attended Hervormd Gymnasium in Amsterdam, a six-year secondary school for gifted students with a curriculum consisting of six mandatory languages (including Greek and Latin), from which he graduated in 1957 with a specialization in the natural sciences and mathematics. After shortly considering to enroll in the college of physical education-Don was an accomplished basketball player-he attended the six-year psychology program at the University of Amsterdam. Don was not very impressed with the curriculum until he met Adriaan de Groot, a towering figure in postwar Dutch psychology and the founder of the university's Psychological Methods group. After his graduation in 1965, Don was appointed as Assistant Professor of Psychological Methods. His first job was at the Department of Exam Techniques, a precursor of the Dutch educational testing bureau Cito (which was also founded by De Groot). This department was headed by Robert van Naerssen, a psychometrician far ahead of his time (van den Brink & Mellenbergh, 1984), whose impact was, however, limited as he published mainly in Dutch. Van Naerssen influenced Don's thinking greatly, and Don would commemorate his ideas and contributions often in conversation. Don defended his Ph.D. thesis, entitled Studies on Educational Tests, in 1971. The dissertation, in which he reported psychometric analyses of the Amsterdam School Test, contains the first documented operational use of the Rasch model in The Netherlands.The next step in his academic career was an appointment as an Associate Professor in the Psychology program at Utrecht University in 1971, with the assignment to develop a new special-Correspondence should be made to Denny Borsboom,

show abstract

Coefficients for Tests from a Decision Theoretic Point of View

Abstract: From a decision theoretic point of view a general coefficient for tests, d, is derived. The coefficient is applied to three kinds of decision situations. First, the situation is considered in which a true score is estimated by a function of the observed score of a vick, 1968, p. 63).

Cited by 17 publications

References 11 publications

Simultaneous Optimization of Decisions Using a Linear Utility Function

Simultaneous Optimization of Decisions Using a Linear Utility Function

The Internal and External Optimality of Decisions Based on Tests

Obituary GIDEON J. MELLENBERGH (1938–2021)

Contact Info

Product

Resources

About