External validity is more than skin deep: Some answers to criticisms of laboratory experiments.

Abstract

We discuss some criticisms of laboratory experiments in psychology, giving special attention to the claim that these experiments lack external validity (since it is -widely assumed that ecological validity makes for external validity). We suggest that representative designs are inadequate for testing causal hypotheses, that ecological validity may facilitate the formulation of population estimates but is not necessary for causal hypothesis testing, and that experiments are not conducted to establish population estimates. Moreover, the meaning that subjects assign to the laboratory setting and their actions, rather than the laboratory setting's mundane realism, affects the generalizability of the laboratory results. We emphasize, as other writers have, that whether laboratory results are generalizable to other situations is an empirical question. Research on aggression, especially in regard to the "weapons ef-. feet," is employed to illustrate the possible extension of laboratory findings to other, more natural, situations.Most criticisms of laboratory experiments in psychology fault them for their "artificiality" or, in more sophisticated terms, for their lack of external validity. As everyone knows, the great majority of psychology's experiments employ a very limited sample of participants (typically, college students) placed in a fairly unique setting (a university laboratory) and usually working on tasks bearing little resemblance to their everyday activities. Given the unrepresentativeness of these subjects and situations, the critics ask, how can the findings be generalized to the "real world" of ordinary people engaged in their daily lives? In one form or another, this question has been raised by professionals and outsiders, by trained social scientists, and by persons having only a slight acquaintance with the field. There can be no easy answer. Anyone fa-

Facing Differences With an Open Mind: Openness to Experience, Salience of Intragroup Differences, and Performance of Diverse Work Groups

“…Although the main aim of experimental studies is not to obtain external validity (Berkowitz & Donnerstein, 1982), reports of experimental research tend to elicit questions of external validity. Obviously, then, confidence in the conclusions advanced here can be strengthened when the current results are replicated in a study of teams in actual organizations, and this would indeed seem an important avenue for future research.…”

Goal importance, self-focus, and the goal-setting process.

“…Further, the significant attrition that will inevitably result in longitudinal research at this level in the retail industry could have affected this already unrepresentative sample in unknown ways. Yet, in studies where the primary interest is in testing theory, representativeness of the sample is of relatively little concern (Berkowitz & Donnerstein, 1982;Mook, 1983). The external validity of these findings can only be established by future research in other settings, for as Cook and Campbell (1979) stated, "external validity is enhanced more by many heterogeneous small experiments, than by one or two large experiments" (p. 80).…”

Trait aggressiveness and hockey penalties: Predicting hot tempers on the ice.

“…Such findings are used to argue that the inventories developed by Buss and his colleagues have predictive validity. Although the current authors accept these laboratory paradigms as valid measures of aggressive behaviors (Anderson & Bushman, 1997; also see Berkowitz & Donnerstein, 1982), others have been less convinced (e.g., Gottfredson & Hirschi, 1993;Kane, Joseph, & Tedeschi, 1976;Tedeschi & Quigley, 1996). Laboratory studies in general are often criticized for their "artificiality."…”