Limitations of the Test Reviewing Process: A Response to Clarizio and Mehrens

Abstract

This is a response to the article by Clarizio and Mehrens in this issue. The author argues that the article by Clarizio and Mehrens is too negative in tone and fails to acknowledge the strengths and values of SOI tests.As research on evaluation and human judgment has shown, evaluations are constrained by the particular experiences and information of any evaluator. To some evaluators, the object of evaluation may be a &dquo;rose bush with thorns&dquo; and, yet, to others it may be a &dquo;thorn bush with roses.&dquo; In the field of test publishing and test reviewing, a review of the literature reveals a veritable obsession with the &dquo;thorns&dquo; among the more rosy aspects of published tests. It was apparently O. K. Buros who set the tone, with near missionary zeal, for the dogged critique of inadequacies in order that the field of test publishing might be improved over time. His mission was and still is admirable. In my opinion, however, it can, in the extreme, degenerate into a negative approach which does not recognize the practical difficulties of developing and standardizing worthwhile clinical instruments.As an example, a review of the Structure of Intellect Learning Abilities Tests (Meeker & Meeker, 1975;Meeker, Meeker, & Roid, 1985) by Clarizio and Mehrens (1985), reflects the zeal of early Buros publications in which test reviewers deluged unsuspecting test developers with negative modifiers. As test authors and professionals within the test publishing industry would attest, some test critiques were based on the reading of a limited set of published materials concerning a test, on limited clinical application of the instruments, or both. Test reviewing is a difficult task and a necessary one, bearing with it the increased responsibility of being objective and fair.In light of the preceding comments, the Clarizio and Mehrens review is quite negative in tone, contains little coverage of the strengths of the SOI-LA tests, approaches the task without the apparent benefit of clinical use of the tests, and compares the tests to inappropriate referents.Each of these characteristics will be discussed briefly, in turn. Strengths of the SOI ApproachThe Structure of Intellect Learning Abilities Tests are, as the name implies, an application of the Guilford ( 1959,1967) &dquo;SI&dquo; model to the assessment of a subset of abilities relevant to learning. There remains, however, an important distinction between the model of Guilford (and the adult-level tests he developed and published) and the SOI-LA application of the model. Strengths of the SOI-LA differ somewhat from those of Guilford's entire model. Thus, when Clarizio and Mehrens (1985) quote Eysenck (1967), Cattell (1971), and others as being critical of the &dquo;SOI,&dquo; they present the inaccurate implication that the full Guilford model and the SOI-LA tests are identical in all respects. The SOI-LA tests, on the contrary, do not attempt to measure any of the Behavioral dimensions in the full Guilford model, for example. When Clarizio and Mehrens ...

A History of Mental Ability Tests and Theories

“…Even though we Mental Ability Tests 87 strongly believe that theory advances science, test developers who invest hundreds of thousands of dollars into test development might be advised that their tests should remain theoreticallylight, so as not to be easily disproven. In a spirited defense of one falsified intelligence test we have not discussed (the Structure of Intellect Learning Abilities tests [Meeker, Meeker, & Roid, 1985]), Roid (1985) rued the tendency of critical test reviewers to "degenerate into a negative approach which does not recognize the practical difficulties of developing and standardizing worthwhile clinical instruments" (p. 121), asking, "Isn't it fair to give time for developing tests (for which a stock of printed materials may already be warehoused), to shift to newly developed techniques?" (p. 122).…”