The Evaluation of Differences in Test Performance of Two or More Groups

Abstract

A two-factor analysis of variance with multiple measurements on one factor was conducted among the 40 items of the Vocabulary test of TOEFL for six language groups. All sources of variance were found to be significant beyond the one per cent level. Of particular interest was the item x group interaction which was examined by analyzing the item difficulty plots for each language group against a spaced sample of all candidates taking this form of the test at its first formal administration. A measure of the deviation of each item from the central tendency of the plot was developed, expressing the degree to which the item was especially difficult or especially easy for a particular language group relative to the other items. A distribution of these measures is given for each of the six language groups.

Comparison of Procedures for Detecting Test-Item Bias with Both Internal and External Ability Criteria

“…Items are considered biased if they are relatively more difficult for one group than for another. Angoff and Ford (1973) introduced the item-difficulty delta plot method (designated TID-Angoff in this paper); it is one of the most common methods in use (Angoff, 1975;Angoff & Modu, 1973;Angoff & Sharon, 1974;Breland, Stocking, Pinchak & Abrams, 1974). Item difficulties or ?…”

Comparison of Procedures for Detecting test-Item Bias with both Internal and External Ability Criteria

“…Items are considered biased if they are relatively more difficult for one group than for another. Angoff and Ford (1973) introduced the item-difficulty delta plot method (designated TID-Angoff in this paper); it is one of the most common methods in use (Angoff, 1975;Angoff & Modu, 1973;Angoff & Sharon, 1974;Breland, Stocking, Pinchak & Abrams, 1974). Item difficulties or £ values (the proportion of examinees getting the item right) are computed separately for each group.…”

Research on Validity Theory and Practice at ETS

“…As noted earlier, from ETS's inception, its research staff had been concerned about fairness issues and had been actively investigating group differences in performance since the 1960s (Angoff and Ford 1973;Angoff and Sharon 1974;Cardall and Coffman 1964;Cleary 1968), but no fully adequate methodology for addressing group differences at the item level had been identified. The need to address the many obstacles facing the effective implementation of DIF was imposed on ETS researchers in the early 1980s:…”