Telling What We Know: The Use of Verbal Report Methodologies in Psychological Research

CAPAS 2.0: A computer tool for coding transcribed and digitally recorded verbal reports

“…Thus CAPAS 2.0 embodies in its design the localized encoding principle (Crutcher, 1994;Crutcher 2003;Crutcher et al, 1994;Ericsson & Simon, 1993). As Ericsson and Simon have emphasized, it is important to address concerns about possible bias in the treatment of verbal report data in order to place verbal report data on par with other types of behavioral data.…”

“…For example, the degree of reliability in applying rules for many of these systems is determined by factors controlled not by the computer system, but by the researcher, factors such as the clarity of the coding rules and how well coders are trained (MacLin & MacLin, 2005). Furthermore, as previous researchers have emphasized, a thorough task analysis prior to constructing a coding scheme is very important (Crutcher, 1994;Crutcher, 2003;Ericsson, 2006;Ericsson & Simon, 1993;Schraagen, 2006).…”

“…One potential solution to this dilemma is to define independent verbal reports or report segments for data from an entire group of participants and present these segments randomly to coders without contextual information such as the condition or trial number associated with a report. This approach of minimizing the context surrounding a report that is being coded is referred to as the localized coding (or encoding) principle (Crutcher, 1994(Crutcher, , 2003Ericsson & Simon, 1993), and it is an important design feature of the system described here.…”

A computer-aided digital audio recording and encoding system for improving the encoding of verbal reports

“…The increasing use of verbal report methodologies requires that methods for collecting, coding, and analyzing these reports must improve accordingly. A number of theoretical issues have been raised by previous researchers, such as the need for rigorous task analysis before reports are collected and the importance of localized encoding schemes, which require that a given protocol be coded with minimum reliance on other protocol segments or surrounding context (Crutcher, 1994;Crutcher, Ericsson, & Wichura, 1994;Ericsson & Simon, 1993). My central concern here is the use of computer hardware and software tools to improve the efficiency and ease of collecting and coding verbal reports while adhering to these important theoretical constraints.…”

“…For example, they provide a high temporal density of behavioral data critical to the evaluation of psychological theories (Anderson, 1987;Crutcher, 1994Crutcher, , 1998Ericsson & Simon, 1993;Simon & Kaplan, 1989). The increasing use of verbal report methodologies requires that methods for collecting, coding, and analyzing these reports must improve accordingly.…”