The Concept of Validity.

Abstract

This article advances a simple conception of test validity: A test is valid for measuring an attribute if (a) the attribute exists and (b) variations in the attribute causally produce variation in the measurement outcomes. This conception is shown to diverge from current validity theory in several respects. In particular, the emphasis in the proposed conception is on ontology, reference, and causality, whereas current validity theory focuses on epistemology, meaning, and correlation. It is argued that the proposed conception is not only simpler but also theoretically superior to the position taken in the existing literature. Further, it has clear theoretical and practical implications for validation research. Most important, validation research must not be directed at the relation between the measured attribute and other attributes but at the processes that convey the effect of the measured attribute on the test scores.

Validation of the Amsterdam Beverage Picture Set: A Controlled Picture Set for Cognitive Bias Measurement and Modification Paradigms

“…We have formulated a position on stimulus validity based on the theory of Borsboom and colleagues (2004), by proposing that stimuli are valid for influencing an attribute if variation in the stimuli cause variation in the attribute. We chose “urge to drink” as our criterion attribute because we presupposed this criterion to be relatively uncontroversial and universal.…”

“…A good starting point may be the position of Borsboom and colleagues (2004, p. 1061), which states that “a test is valid for measuring an attribute if (a) the attribute exists and (b) variations in the attribute causally produce variations in the measurement outcomes.” This position applies well to the kind of response time (RT) tasks used for measuring cognitive biases: substance‐related and substance‐unrelated stimuli are presented to a participant, causing variation in a mental construct, which then is measured. For example, in an alcohol visual probe task, we presuppose the existence of an attribute named selective attention for alcohol.…”

Implicit measures: A normative analysis and review.

“…This is, however, not entirely true. As argued by Borsboom et al (2004) and Tuerlinckx et al (2002), a measure can be valid (i.e., caused by the to-be-measured attribute) even when it is not reliable. Such a situation can, for instance, arise when the underlying psychological attribute does not remain stable over time or context.…”

“…Whereas Borsboom et al (2004) promoted experimental studies as the primary way to study validity, until now, the correlational approach has dominated validation research. Borsboom et al ar-gued that the correlational approach as typically adopted in validation research is suboptimal for the study of the validity of measures because correlational evidence (a) does not allow for causal inferences and (b) is not directed at examining the relation between psychological attributes and measurement outcomes.…”

“…A final important point in the work of Borsboom et al (2004) is that they distinguished between the validity of a measure and its overall quality. They argued that a valid measure is not necessarily reliable or predictive of criterion variables and could even measure different attributes in different groups of respondents (Borsboom et al, p. 1070).…”