Considering the Preponderance of Evidence: A Reply to Cummings

Abstract

Cummings's 1985 response to our prior article suggests that the American Psychological Association was justified in not adequately describing or qualifying medical-cost-offset research findings, because the “preponderance of evidence” in the over-all literature is “quite persuasive.” We argue in return that one's willingness to accept a body of evidence as persuasive does not excuse one from providing important qualifications for findings, especially when significant threats to validity exist. We question whether the evidence is indeed persuasive. A review of the evidence suggests that the potential for serious regression and selection biases is not limited to the studies described in the APA document. Cummings acknowledges that studies of offset have weaknesses but argues that collectively they provide evidence analogous to that establishing a causal link between smoking and disease. On the contrary, as we describe, the evidence linking smoking and disease is vastly stronger. We do not find compelling the analogy pairing the Surgeon General's conclusion about smoking with a government agency's report concluding that psychological care reduces medical costs. We appreciate that one would not want to use a no-treatment control group in the kind of research Cummings is currently conducting, but there may be alternative, quasi-experimental designs which allow testing for absolute effects while controlling for possible regression.