An Investigation of the Simultaneous Moderation of Average Gender and African‐american Score Differences on a Test of Mathematical Reasoning

Abstract

A pool of items from operational tests of mathematical reasoning was constructed to investigate the feasibility of using automated test assembly methods to simultaneously moderate possibly irrelevant differences between the performance of women and men and African-American and White test takers. None of the artificial tests investigated exhibited substantial impact moderation, although the estimated mean scaled score differences for the relevant population indicated a modest move in the intended direction: the difference between scaled score means was reduced by about 20% for women and men and about 9%for African-American and White test takers. Although many issues in the implementation of this methodology remain to be solved, the consideration of impact in automated test assembly along with the maintenance of the detailed test plan appears to be a potential method of moderating possibly irrelevant mean test score differences.

An Empirical Investigation of Impact Moderation in Test Construction

“…This suggestion was based on the observation that moderated impact tests tend to have easier and harder items than tests constructed ignoring impact. For gender impact, where the sample sizes of the two groups are roughly equal, there is a mathematical basis for this observation, as demonstrated in Stocking et. al, (1998b).…”

An Empirical Investigation of Impact Moderation in Test Construction

“…In addition, a larger average standard error of measurement can be expected for tests designed to have moderated impact when compared to tests assembled without regard to impact. The mathematical proofs underlying these assertions are given in Stocking, Jirele, Lewis, and Swanson (1998b). The right most column of Table 6 displays the concurrent validity, that is the correlation of test scores with self-reported academic grade point average.…”