Threats to generalization in the use of elicited, purloined, and contrived messages in human communication research

Abstract

This paper discusses threats to valid generalization in the study of messages, focusing upon problems of external validity. Three classes of messages are examined and the special threats associated with each are described. Additionally, general problems which cut across these classes are indicated. It is suggested that replication of research efforts is the best procedure for exploring the limits of message generalization and that the discovery of such limits can be theoretically fruitful. KEY CONCEPTS Message effects, generalization of results, validity of inferences, research design. JAMES J. BRADAC is Professor and Chair in the

The design and analysis of language experiments in communication

“…Is there a message that has not been manipulated-by someone, be it an experimenter or someone else? Bradac (1986) touches on this point when he raises the question of "irrelevant awareness" of sources who generate messages. He suggests that nonexperimenters may be somewhat conscious of subsequent scrutiny by communication scholars and the like.…”

“…Jackson (1984) argued that ubiquitous interactions will be created by "unexamined biases operating" in designs in which the experimenter creates all the messages (p. 9). Bradac (1986) refers to this as the "idiosyncratic warping" problem. Jackson speculates that such warping exists, but presents no empirical support for this claim.…”

“…They argued that it is good to use natural messages and bad to use artificial messages. Less polemic language was used by Bradac (1986), who used the labels "contrived versus natural." We prefer the labels "controlled and uncontrolled" messages for the same distinction.…”

Experimental Methodology in Journalism and Mass Communication Research

“…The arguments presented here are informed by similar arguments found elsewhere. 22 There are two kinds of variance that are important in experiments: treatment variance and message variance. When researchers are interested in how people respond to media messages, treatment variance refers to the message manipulations, that is, how the levels of each independent variable vary in a message.…”

Changing Conceptions of "Message Effects" A 24-Year Overview

“…Jackson (1992) noted that samples of message replications generated by a researcher can be sources of invalidity due to bias (e.g., a choice of messages partial to the hypothesis) or collection-category mismatch (e.g., failing to represent the full range of message variability). Bradac (1986) argued that "elicited, purloined, and contrived messages" can fail to generalize to a population of messages for a variety of reasons, including "idiosyncratic warping" (p. 60) (e.g., tendencies of the researcher to unconsciously choose similar style or language in constructing messages).…”