Difficulty and Other Correlates of Criticalness Response Style at the Item Level1

Abstract

This study examined the.relationship between the difficulty of items on an achievement-type test of report writing and (a) the items' correlations with the test's criticalness response style score, and (b) the items' correlations with the test's content score. Item readability and item format variables were also analyzed. The major findings were that (a) the items' difficulty was related to their correlations with the content score and unrelated to the items' correlations with the response style score; (b) the items' location in the test was generally related to their correlations with the response style score, and unrelated to the other variables; and (c) the readability indexes were not related to the three basic item variables. DIFFICULTY AND OTHER CORRELATES OF CRITICALNESS RESPONSE STYLE AT THE ITEM LEVEL 1Largely as a result of Cronbach's (1946; reviews of the early literature on response sets--or response styles--on tests, the view is widely held that tests and test items that are difficult or ambiguous are most affected by those response styles, such as acquiescence, evasiveness, and extremeness, that are linked to the response format.Despite the prevalence of this view, the results of relevant studies are not altogether convincing, and are largely limited to two response styles--acquiescence, and the position of the chosen alternative on multiple-choice tests.One major group of relevant studies examined acquiescence on tests or items of varying difficulty or ambiguity. Only one of the studies (Gage, Leavitt, & Stone, 1957) concerns the difficulty issue. In a comparison of difficult and easy general information items, the reliability of the number of true responses was .68 for the 50 difficult items, and .09 for the 40 easy ones. The two kinds of items, however, did not represent exactly the same content areas.The other studies of this kind concern various facets of item ambiguity. Two of these studies directly concern ambiguity. In one study (Bass, 1955), the original and reversed version of the California F scale items that were rated most opposite in meaning (and, hence, assumed to be clear in meaning) appeared to measure content (i.e., the correlation between the two items was relatively high and in the content direction), while those rated less opposite in meaning (and, hence, assumed to be -2-relatively ambiguous in meaning) did not, evidently because of the operation of acquiescence on the latter items. Moreover, an analysis of variance found significantly (£ < .01) greater acquiescence to the pairs of items that were least opposite in meaning. The assumption in this study that the rated level of opposition measures ambiguity is questionable, for it is possible for a pair of items that are rated as similar in meaning to be very upambiguous. In a second study (l3a.nta, 1961), the number of agree responses on attitude scales increased and the number of extreme responses decreased with the ambiguity of the scales' referents. The referents were "President Eisenhower," "College Fraternities,"...