The process of making judgments and decisions requires a method for combining data. To compare the accuracy of clinical and mechanical (formal, statistical) data-combination techniques, we performed a meta-analysis on studies of human health and behavior. On average, mechanical-prediction techniques were about 10% more accurate than clinical predictions. Depending on the specific analysis, mechanical prediction substantially outperformed clinical prediction in 33%-47% of studies examined. Although clinical predictions were often as accurate as mechanical predictions, in only a few studies (6%-16%) were they substantially more accurate. Superiority for mechanical-prediction techniques was consistent, regardless of the judgment task, type of judges, judges' amounts of experience, or the types of data being combined. Clinical predictions performed relatively less well when predictors included clinical interview data. These data indicate that mechanical predictions of human behaviors are equal or superior to clinical prediction methods for a wide range of circumstances.
Given a data set about an individual or group (e.g., interviewer ratings, life history or demographic facts, test results, self-descriptions), there are two modes of data combination for a predictive or diagnostic purpose. The clinical method relies on human judgment that is based on informal contemplation and, sometimes, discussion with others (e.g., case conferences). The mechanical method involves a formal, algorithmic, objective procedure (e.g., equation) to reach the decision. Empirical comparisons of the accuracy of the two methods (136 studies over a wide range of predictands) show that the mechanical method is almost invariably equal to or superior to the clinical method: Common antiactuarial arguments are rebutted, possible causes of widespread resistance to the comparative research are offered, and policy implications of the statistical method's superiority are discussed. In 1928, the Illinois State Board of Parole published a study by sociologist Burgess of the parole outcome for 3,000 criminal offenders, an exhaustive sample of parolees in a period of years preceding. (In Meehl 1954/1996, this number is erroneously reported as 1,000, a slip probably arising from the fact that 1,000 cases came from each of three Illinois prisons.) Burgess combined 21 objective factors (e.g., nature of crime, nature of sentence, chronological age, number of previous offenses) in unweighted fashion by simply counting for each case the number of factors present that expert opinion considered favorable or unfavorable to successful parole outcome. Given such a large sample, the predetermination of a list of relevant factors (rather than elimination and selection of factors), and the absence of any attempt at optimizing weights, the usual problem of cross-validation shrinkage is of negligible importance. Subjective, impressionistic, "clinical" judgments were also made by three prison psychiatrists about probable parole success. The psychiatrists were slightly more accurate than the actuarial tally of favorable factors in predicting parole success, but they were markedly inferior in predicting failure. Furthermore, the actuarial tally made predictions for every case, whereas the psychiatrists left a sizable fraction of cases undecided. The conclusion was clear that even a crude actuarial method such as this was superior to clinical judgment in accuracy of prediction. Of course, we do not know how many of the 21 factors the psychiatrists took into account; but all were available to them; hence, if they ignored certain powerful predictive factors, this would have represented a source of error in clinical judgment. To our knowledge, this is the earliest empirical comparison of two ways of forecasting behavior. One, a formal method, employs an equation, a formula, a graph, or an actuarial table to arrive at a probability, or expected value, of some outcome; the other method relies on an informal, "in the head," impressionistic, subjective conclusion, reached (somehow) by a human clinical judge.
Using the Scale for the Assessment of Thought, Language, and Communication (TLC), we examined the frequency of "thought disorder" in 94 normal volunteers and 100 psychiatric patients (25 each suffering from manic disorder, schizoaffective disorder, schizophrenic disorder, disorganized type, and schizophrenic disorder, paranoid type). We observed the manics to have a substantial amount of thought disorder and the normals to have a modest amount, suggesting that thought disorder is probably not pathognomonic of schizophrenia. The patients with affective illness did, however, show a somewhat different pattern of abnormality. In particular, patients with affective psychosis have more prominent positive thought disorder, while the schizophrenic patients tend to have more negative thought disorder. Evaluation of the patients 6 months later indicated that most types of thought disorder remit in the manics, while they persist in the schizophrenics; patients with schizoaffective disorder also tend to improve substantially. The strongest predictor of outcome was the presence of negative thought disorder.
The effects of changes in depression-relevant cognition were examined in relation to subsequent change in depressive symptoms for outpatients with major depressive disorder randomly assigned to cognitive therapy (CT; n = 32) versus those assigned to pharmacotherapy only (NoCT; n = 32). Depression severity scores were obtained at the beginning, middle, and end of the 12-week treatment period, as were scores on 4 measures of cognition: Attributional Styles Questionnaire (ASQ), Automatic Thoughts Questionnaire (ATQ), Dysfunctional Attitudes Scale (DAS), and the Hopelessness Scale (HS). Change from pretreatment to midtreatment on the ASQ, DAS, and HS predicted change in depression from midtreatment to posttreatment in the CT group, but not in the NoCT group. It is concluded that cognitive phenomena play mediational roles in cognitive therapy. However, data do not support their status as sufficient mediators.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.