This article examines Ochoa and Olivarez's (1995)replication study of Swanson and Malone's (1992) meta-analysis of sociometric research for children with learning disabilities. The two research syntheses agree in direction and outcomes, but vary substantially in identifying the moderator variables that underlie effect sizes. Some of the differences between the two syntheses were related to the effects of gender, ethnicity, and type of measurement on effect size. Differences between the two syntheses were attributed to (a) inadequate reporting of coding reliability, (b) failure to include similar articles for analysis, and (c) poor operationalization of the term learning disabilities as the basis for article selection. Because differences existed in coding schemes and effect sizes between the two syntheses, guidelines for enhancing replication for future syntheses are suggested. These guidelines reflect several criteria on which to judge a meta-analysis of the literature.
Syntheses of research-especially meta-analyses-on children with learning disabilities (LD) appear quite frequently in the literature. Unfortunately, there have been very few criteria to judge the quality of these reports. In addition, there have been few if any replications, to my knowledge, on the findings of these syntheses. Recently, Ochoa and Olivarez (1995) conducted a replication of Swanson and Malone's (1992) meta-analysis of sociometric research on students with learning disabilities. Earlier, Swanson and Malone (1992) had found that students with learning disabilities had considerably lower sociometric status than their nondisabled counterparts. Ochoa and Olivarez's replication supported Swanson and Malone's findings in terms of the direction and magnitude of the effects. Ochoa and Olivarez argued, however, that their study was more accurate in terms of (a) effect sizes (ES's), (b) conceptual comparability of dependent measures, and (c) internal validity. In sum, they argued that (a) their ES's were smaller and took into consideration sample size and outliers, (b) they correctly classified measures, and (c) they placed parameters on the quality of the studies included in the synthesis. Because of a failure to replicate our findings completely, our natural response was to elaborate and qualify our synthesis. Much to our surprise, however, we found the meta-analysis by Ochoa and Olivarez more thorough and methodologically sophisticated than ours. We have some quibbles with Ochoa and Olivarez (1995), however, such as their failure to (a) report interrater agreement on the various codes, (b) define what is meant by a "methodologically sound study," and (c) compare conclusions on various categories on the same articles (the percentage of overlap in articles