An Improved Design for Examining the Importance of Context Dependence

Abstract

Few investigators have sought to determine, rather than merely to assume, the importance of context dependence. It is contended that those who have empirically studied the issue have not addressed the most appropriate research question. The rationale of a research design for estimating the importance of context dependence is presented. Two pilot studies are provided to illustrate the method of investigating the importance of context dependence and of comparing aspects of utility and validity of competing passage dependence indices.In recent years a body of literature has accumulated concerning passage dependence in reading comprehension and the more general case of context dependence (CD) in any kind of interpretive exercise type test. Writers have offered alternative methods of operationalizing CD Although questions regarding the importance of CD are amenable to empirical research, few investigators have sought to determine, rather than to assume, its importance. This article concerns the practical importance of CD. The purposes were to (1) propose a research design for estimating for a given population the extent to which items with high CD measure a construct that is independent from that measured by items with low CD, (2) show how this design can be used empirically to compare the utility of alternate CD indices, and (3) provide findings of two explorative, demonstrative studies. RATIONALEPrevious studies of the importance of CD have secured scores for examinees administered a set of questions without the context upon which they are based and at University of Manitoba Libraries on June 9, 2015 jlr.sagepub.com Downloaded from

An Empirical Evaluation of Three Definitions of Context Dependence

“…The trivial extent to which these are less than unity indicates that the trait measured by questions with good Tuinman indices seems to differ little from that measured by questions with poor Tuinman indices. These findings are consistent with those of one of Hanna's (1979) demonstration studies, but in the other earlier study, the discriminant validity of Tuinman's constructs of high and low CD was much better. For the Hanna-Oaster CD definition, the corrected correlation between scores of items with the best and worst values are .93 and .95.…”

“…With the content of this reading comprehension test in the samples used for its national standardization, the attribute measured by items having higher Pyrczak indices was virtually identical to (or at least was interchangeable with) that measured by items having lower indices; the subscores correlated almost as highly as their reliabilites would allow. These findings are consistent with those of minor previous studies with tests of reading and social studies (Hanna, 1979); all four sets of data failed to demonstrate discriminant validity of Pyrczak's constructs of high and low CD.…”

“…For the Hanna-Oaster CD definition, the corrected correlation between scores of items with the best and worst values are .93 and .95. These findings were not as favorable to the discriminant validity of the Hanna-Oaster constructs of high and low CD as were the findings of the earlier (Hanna, 1979) pilot studies.…”

“…Since they have been discussed and compared by their authors (Hanna & Oaster, 1978-1979Pyrczak, 1975Pyrczak, -1976Tuinman, 1973Tuinman, -1974 and since this study concerns empirical rather than rational comparisons, their relative merits will not be treated. The indices themselves were recently republished and described in this journal (Hanna, 1979).…”

“…Only two minor studies with junior high school students (Hanna, 1979) have addressed the topic this way. These pilot investigations were marred in two ways.…”