Problematic Phrases in the Conclusions of Published Outcome Studies

Abstract

Objective: This study examined the extent to which conclusions of published outcome studies contain phrases that could be misconstrued as implying more empirical support than is warranted. Methods: All articles (N = 138) reporting outcome studies from 2000 to 2005 in two social work research journals and two topical database searches were assessed regarding research design, findings, and wording of conclusions. Substantial interrater agreement was indicated by kappa values of .95 for research design, 1.00 for nature of findings, and .70 for wording of conclusions. Results: Of the articles, 70% used designs that do not warrant making conclusive causal inferences, and 60% of articles with those designs contained phrases that could be misconstrued or exploited as implying an inflated evidence-based status. Conclusion: To prevent evidence-based practice from becoming a meaningless shibboleth, authors, reviewers, and editors should become vigilant in avoiding wording that could be misconstrued as implying more empirical support than is warranted.

What Types of Designs are We Using in Social Work Research and Evaluation?

“…They included Journal of Social Service Research (SSR), Research on Social Work Practice (RSWP), and Social Work Research (SWR). The latter two were recently used by Rubin and Parrish (2007) in their assessment of published outcomes studies and EBP, who also assessed other elements of design in their study. …”

“…Despite the fact that Rubin and Parrish (2007) selected studies for their review of N ¼ 138 from two journals, plus electronic sources related to evaluating the effectiveness of interventions, programs, or policies (a much narrower evaluative orientation), they found that only 20% used randomized experimental studies, and another 9% used quasiexperimental designs with good controls for potential selectivity biases (p. 337), thus offsetting this important internal validity threat.…”

Social Work Education and Avoidable Ignorance

“…Avoidable distortions of views are common. Professionals are often bamboozled by false claims about what "we know" in professional journals and textbooks as well as in the media (e.g., Rubin & Parrish, 2007). A bogus claim is one that misrepresents the evidentiary status of the claim.…”

Celebrating Our 25th Anniversary

“…Mixed methods papers involving quantitative analyses should have this element of the article compliant with these guidelines. Causal inferences, if any, should be made conservatively and not go beyond the limits imposed by the presented methods and data (see Rubin & Parrish, 2007). In addition, the abstracts of data-based articles must include the following headings in boldface: Purpose:, Methods:, Results:, and Conclusions:.…”