A nonparametric approach for assessing latent trait unidimensionality

Stout, William

doi:10.1007/bf02294821

Cited by 406 publications

(464 citation statements)

References 24 publications

Supporting

Mentioning

446

Contrasting

Unclassified

Order By: Relevance

“…Stout (1987Stout ( , 1990, Zhang (1996), and Zhang and Stout (1996)), and from a parametric factor analytic approach (e.g. Hulin, Drasgow, and & Parsons 1983;McDonald 1981McDonald , 1982 has been written on detecting conditional independence among items and/or assessing the number of latent dimensions.…”

Section: Introductionmentioning

confidence: 99%

A Bayesian Random Effects Model for Testlets

Bradlow¹,

Wainer²,

Wang³

1998

ETS Research Report Series

105

135

View full text Add to dashboard Cite

Standard item response theory (IRT) models fit to dichotomous examination responses ignore the fact that sets of items (testlets) often come from a single common stimuli (e.g. a reading comprehension passage). In this setting, all items given to an examinee are unlikely to be conditionally independent (given examinee proficiency). Models that assume conditional independence will overestimate the precision with which examinee proficiency is measured. Overstatement of precision may lead to inaccurate inferences such as prematurely ending an examination in which the stopping rule is based on the estimated standard error of examinee proficiency (e.g an adaptive test). To model examinations that may be a mixture of independent items and testlets, we modified one standard IRT model to include an additional random effect for items nested within the same testlet. We use a Bayesian framework to facilitate posterior inference via a Data Augmented Gibbs Sampler (DAGS, Tanner and Wong 1987). The modified and standard IRT models are both applied to a data set from a disclosed form of the SAT. We also provide simulation results that indicates that the degree of precision bias is a function of the variability of the testlet effects, as well as the testlet design. Rubin for their helpful suggestions and comments, Ida Lawrence and Miriam Feigenbaum for providing us with the SAT data analyzed in Section 6, and to the two anonymous referees for their careful reading and thoughtful suggestions on an earlier draft. We are also grateful to the Educational Testing Service for providing the resources to do this research.1

show abstract

Section: Introductionmentioning

confidence: 99%

A Bayesian Random Effects Model for Testlets

Bradlow¹,

Wainer²,

Wang³

1998

ETS Research Report Series

105

135

View full text Add to dashboard Cite

show abstract

“…DIMTEST has been found to discriminate between unidimensional and two-dimensional tests for a variety of simulated test data when the correlation between abilities is as high as .7 (Stout, 1987; Nandakumar & Stout, in press). Nandakumar (1991)…”

Section: Assessing Dimensionality-comparisonmentioning

confidence: 99%

“…If EO is sought, on the other hand, it is recommended that, at most, one-quarter of the total items should be selected that tap the same ability. After selecting items of AT1, items of AT2 are selected, also of the same size M, so that items of AT1 and AT2 have the same difficulty distribution (for details see Stout, 1987). The remaining items (n=N-2M) form the partition subtest PT.…”

Section: This Can Be Achieved Either Through Factor Analysis (Fa) Ormentioning

confidence: 99%

“…When the model underlying the test item responses is close to essentially unidimensional, items of AT1, AT2, and PT would all be of the same dominant dimension; therefore, the value of the statistic T computed based on AT1, AT2 would be "small," leading to the tenability of H 0 . When the model underlying the test responses is not essentially unidimensionaJ, however, items of AT1 would be dimensionaly different from items of AT2 and PT and the value of the statistic T will be "large" leading to the rejection of H 0 .DIMTEST has been found to discriminate between unidimensional and two-dimensional tests for a variety of simulated test data when the correlation between abilities is as high as .7 (Stout, 1987; Nandakumar & Stout, in press). Nandakumar (1991)…”

mentioning

confidence: 99%

“…The basic rationale for the H&R approach is to reject the locally independent, monotone, unidimensional model if the conditional covariances are significantly negative. By contrast, DIMTEST rejects the essentially independent, monotone, essentially unidimensional model if the conditional covariances are significantly positive (it can be shown that the expected value of the numerator of Stout's statistic T is mathematically equivalent to average conditional covariances among AT1 items, Stout (1987)). This apparent contradiction in the criterion for assessing unidimensionality may be resolved by noting the subtle difference in item pair covariances under consideration.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Assessing Dimensionality of a Set of Items - Comparison of Different Approaches

Nandakumar¹

1992

View full text Add to dashboard Cite

ADA25 ?4 oe907Form ApprovedREPORT DOCUMENTATION PAGE PuOic reporting burden for this collection of information IS estimated tO average I hour oer resprise, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments re arding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden. Assessing Dimensionaity of a Set of Items-Comparison of Different Approaches AbstractThis study examines the performance of the following four methodologies for assessing unidimensionality: DIMTEST, Holland and Rosenbaum's approach, linear factor analysis, and nonlinear factor analysis. Each method is examined and compared with other methods on simulated data sets and on real data sets. Seven data sets, all with 2000 examinees, were generated: three unidimensional, and four two-dimensional data sets. Two levels of correlatiotL between abilities were considered: p=.3 and p=.7. Eight different real data sets were used: four of them were expected to be unidimensional, and the other four were expected to be two-dimensional. Findings suggest that, while the linear factor analysis often overestimated the number of underlying dimensions, the other three methods correctly confirmed unidimensionality but differed in their ability to detect lack of unidimensionality. DIMTEST showed excellent power in detecting lack of unidimensionality; Holland and Rosenbaum's and nonlinear factor analysis approaches showed good power, provided the correlation between abilities was low.Subject terms: DIMTEST, unidimensionality, essential dimensionality, non-linear factor analysis, item response theory.A.ce'ion -or ; NTIS C 2,.6I;. .. DTIC QUALITY INSPECTRD 3 K:". Assessing Dimensionality-ComparisonIt is well known that most item response theory (IRT) models require the assumption of unidimensionality. According to Lord and Novick (1968), dimensionality is defined as the total number of abilities required to satisfy the assumption of local independence. If there is only one ability affecting the responses of a set of items to meet the assumption of local independence, then that set is referred to as a unidimensional set.It has also been long argued that responses to test items are multiply determined (Humphreys, 1981(Humphreys, , 1985(Humphreys, , 1986 Hambleton & Swaminathan, 1985, chap. 2;Reckase, 1979Reckase, , 1985Stout, 1987;Traub, 1983;Yen, 1985), and several abilities unique to items or common to relatively few items are inevitable. The ability which the test is intended to measure (i.e., the ability common to all items) will be referred to as the dominant ability, and abilities unique to or influencing responses to few items will be referred to as minor abilities. Given that item responses are multiply determined, it is intuitively clear that, in order to satisfy the assumption of unidimensionality, it is required that a given test measure a single domi...

show abstract