Signal‐detection Properties of Verbal Self‐reports

Abstract

The bias (B'11) and discriminability (A') of college students' self-reports about choices made in a delayed identity matching-to-sample task were studied as a function of characteristics of the response about which they reported. Each matching-to-sample trial consisted of two, three, or four simultaneously presented sample stimuli, a 1-s retention interval, and two, three, or four comparison stimuli. One sample stimulus was always reproduced among the comparisons, and choice of the matching comparison in less than 800 ms produced points worth chances in a drawing for money. After each choice, subjects pressed either a "yes" or a "no" button to answer a computer-generated query about whether the choice met the point contingency. The number of sample and comparison stimuli was manipulated across experimental conditions. Rates of successful matching-to-sample choices were negatively correlated with the number of matching-to-sample stimuli, regardless of whether samples or comparisons were manipulated. As in previous studies, subjects exhibited a pronounced bias for reporting successful responses. Self-report bias tended to become less pronounced as matching-tosample success became less frequent, an outcome consistent with signal-frequency effects in psychophysical research. The bias was also resistant to change, suggesting influences other than signal frequency that remain to be identified. Self-report discriminability tended to decrease with the number of sample stimuli and increase with the number of comparison stimuli, an effect not attributable to differential effects of the two manipulations on matching-to-sample performance. Overall, bias and discriminability indices revealed effects that were not evident in self-report accuracy scores. The results indicate that analyses based on signal-detection theory can improve the description of correspondence between self-reports and their referents and thus contribute to the identification of environmental sources of control over verbal self-reports.

Differential latency and selective nondisclosure in verbal self-reports

“…In particular, signal-detection analyses (Green & Swets, 1966) revealed in verbal self-reports several orderly patterns that were not apparent in the more traditional dependent measure of percent correct (e.g., Critchfield, 1993aCritchfield, , 1993bCritchfield, , 1994Critchfield, , 1996. Signal-detection analysis evaluates the correspondence between two classes of events, normally subject reports and a target stimulus event.…”

“…As applied here, the analysis focuses on the proportion of responses falling into categories defined as in Figure 2. The present data were collected as part of a previously-described DMTS RESPONSE Successful Unsuccessful experiment (Critchfield, 1993a). In each condition of that study, rates of self-reports in the categories shown in Figure 2 were used to calculate indices of self-report bias and discriminability, which revealed a variety of effects not evident in the more global measure of self-report accuracy.…”

“…Following each response, a computer-presented query asked whether the response had been successful, and subjects replied by pressing a "Yes" or "No" button. Experiment 1 analyzed self-report latencies from a previously-published study (Critchfield, 1993a). Latencies generally were longer for self-reports of failure than for selfreports of success.…”

“…Preliminary indications are that they do. For example, the extent and direction of bias in self-reports appears to be a function of the frequency of occurrence of the referent response (e.g., Critchfield, 1993aCritchfield, , 1994Critchfield, , 1996Critchfield & Perone, 1993), just as bias in reports about external stimuli is a function of stimulus frequency (e.g., Craig, 1979;McCarthy & Davison, 1981). In this sense, verbal behavior was found to function similarly to other kinds of behavior, as Skinner supposed.…”

“…Just before the start of the first experimental session, subjects signed an informed consent agreement and read printed instructions. Those used in Experiment 1 have been described previously (Critchfield, 1993a), but were similar to those used in Experiment 2, reproduced in Appendix A. An eight-session preliminary training phase then was used to familiarize subjects with the DMTS task.…”