Many researchers have suggested that the main cause of item bias is the misspecification of the latent ability space, where items that measure multiple abilities are scored as though they are measuring a single ability. If two different groups of examinees have different underlying multidimensional ability distributions and the test items are capable of discriminating among levels of abilities on these multiple dimensions, then any unidimensional scoring scheme has the potential to produce item bias. It is the purpose of this article to provide the testing practitioner with insight about the difference between item bias and item impact and how they relate to item validity. These concepts will be explained from a multidimensional item response theory (MIRT) perspective. Two detection procedures, the Mantel‐Haenszel (as modified by Holland and Thayer, 1988) and Shealy and Stout's Simultaneous Item Bias (SIB; 1991) strategies, will be used to illustrate how practitioners can detect item bias.

Many educational and psychological tests are inherently multidimensional, meaning these tests measure two or more dimensions or constructs. The purpose of this module is to illustrate how test practitioners and researchers can apply multidimensional item response theory (MIRT) to understand better what their tests are measuring, how accurately the different composites of ability are being assessed, and how this information can be cycled back into the test development process. Procedures for conducting MIRT analyses–from obtaining evidence that the test is multidimensional, to modeling the test as multidimensional, to illustrating the properties of multidimensional items graphically‐are described from both a theoretical and a substantive basis. This module also illustrates these procedures using data from a ninth‐grade mathematics achievement test. It concludes with a discussion of future directions in MIRT research.

No abstract

This paper illustrates how graphical analyses can enhance the interpretation and understanding of multidimensional item response theory (IRT) analyses. Many unidimensional IRT concepts, such as item response functions and information functions, can be extended to multiple dimensions; however, as dimensionality increases, new problems and issues arise, most notably how to represent these features within a multidimensional framework. Examples are provided of several different graphical representations, including item response surfaces, information vectors, and centroid plots of conditional two-dimensional trait distributions. All graphs are intended to supplement quantitative and substantive analyses and thereby assist the test practitioner in determining more precisely such information as the construct validity of a test, the degree of measurement precision, and the consistency of interpretation of the numbercorrect score scale. Index terms: dimensionality, graphical analysis, multidimensional item response theory, test analysis. Most psychological and educational tests measure, to different degrees, multiple traits or trait composites. As such, test practitioners must establish the construct validity of each test and subsequently provide users with an interpretation of what the test measures. If a test measures several traits simultaneously, questions that need to be addressed include: (1) what composite of traits is being measured? (2) of the traits being measured, which are primary (i.e., intended to be measured) and which are secondary (i.e., not intended to be measured)? (3) how accurately are each of the various composites being assessed? (4) what is the correct interpretation of the number-correct (or standard) score scale? (5) is this interpretation consistent throughout the entire number-correct score range, or do low scores reflect levels of one composite trait and high score levels reflect another composite trait? and (6) do the secondary traits result in differential performance between identifiable groups of examinees? Typically, item, test, and differential item functioning (DIF) analyses are conducted after every administration of a standardized test. The purpose of this paper is to present a series of graphical representations of multidimensional analyses that can supplement these analyses. The goal of pictorially representing quantitative results is to help the practitioner gain insight into the measurement process and, thus, strengthen the relationship between the test construction process and tl~e quantitative analysis of test results. Graphical analyses serve several functions: 1. They provide a visual perspective that can triangulate or cross-validate traditional quantitative item, test, and DIF analyses. 2. They help measurement specialists gain a better conceptual understanding of the principles of measurement as they apply to a test. 3. They strengthen the link between quantitative analyses and substantive interpretations of what a test is measuring and how well it measures. 4. They can be ...

The characteristics of unidimensional ability estimates obtained from data generated using multidimensional compensatory models were compared with estimates from noncompensatory IRT models. Reckase, Carlson, Ackerman, and Spray (1986) reported that when a compensatory model is used and item difficulty is confounded with dimensionality, the composition of the unidimensional ability estimates differs for different points along the unidimensional ability (&thetas;) scale. Eight datasets (four compensatory, four noncompensatory) were generated for four different levels of correlated two-dimensional &thetas;s. In each dataset, difficulty was confounded with dimensionality and then calibrated using LOGIST and BILOG. The confounding of difficulty and dimensionality affected the BILOG calibration of response vectors using matched multidimensional item parameters more than it affected the LOGIST calibration. As the correlation between the generated two-dimensional &thetas;s increased, the response data became more unidimensional as shown in bivariate plots of the mean & t h e t a s ; 1 as opposed to the mean of & t h e t a s ; 2 for specified unidimensional quantiles.