We examine Graham's (1995) Graham's (1995) response to our comment provides some important addenda to both our meta-analysis (Cooper & Dorr, 1995) and her original box score review of research on race differences in motivation (Graham, 1994). With most of her points, as with her initial review, we have no contention. However, we think a few corrections should be made to her new exposition, both in regard to our work and to meta-analysis in general.
The Issue of Quality ControlGraham states that meta-analysts possess a "reluctance to distinguish between low-and high-quality studies" (p. 509) and "tend to downplay the question of poor-quality studies as a barrier to a good review" (p. 511). We disagree. Rather, we would suggest that many meta-analysts (but not all and not in every circumstance; see, for example, Ottenbacher & Cooper, 1983) eschew the process of discarding studies from consideration based on unitary, dichotomous judgments of quality. Underpinning this decision is a healthy skepticism about the reliability and objectivity of global judgments of research quality. Instead, these metaanalysts assess empirically the impact of multiple dimensions of methodology on the outcomes of studies, and thereby display a great sensitivity to issues of research quality.Regardless, the issue of quality control is moot in the present instance. In our meta-analysis, the evidential base was identical to that constructed by Graham herself. Her quality criteria were our quality criteria. Further, we examined the 515