Train-to-Code: An adaptive expert system for training systematic observation and coding skills

Abstract

Problems in training behavioral observers to a high degree of interindividual accuracy and intraindividual stability are fundamental concerns in descriptive research, as well as in provisions of behavioral intervention services. This article presents design characteristics of and results from three formative evaluations of an adaptive computerized expert system that shapes observation and recording skills and maximizes both individual coding accuracy and stability. The system, called Train-to-Code, allows instructors or trainers to import their own video source files and to code those videos using any appropriate descriptive behavioral-coding scheme. This generates customized expert reference data that automate subsequent training on the basis of an operant response-shaping instructional design model. Successful training relies on transitions through alternative levels of prompting and feedback designed to optimize ongoing performance until stable expert-equivalent levels of interobserver accuracy are maintained without prompting or feedback.

Using Calibration and Interobserver Agreement Algorithms to Assess the Accuracy and Precision of Data From Electronic and Pen‐and‐paper Continuous Recording Methods

“…However, now, like four decades ago, there is still a lack of information in research articles on how observers are trained (Wildman et al, ). This lack of information combined with a lack of research specifically focused on training observers (although see Bass, ; Ray & Ray, ; Wildman et al, ) highlights an area in need of future research if we are to have observation systems that produce quality data.…”

“…Error in sensitivity could also occur in the observer, and so, it is important that the observers be trained to identify and record behaviors across the range of rates that may be observed. However, there has been little research on training observers (Bass, ; Ray & Ray, ), and research reports seldom give detail on observer training (Wildman, Erickson, & Kent, ).…”

“…has been little research on training observers (Bass, 1987;Ray & Ray, 2008), and research reports seldom give detail on observer training (Wildman, Erickson, & Kent, 1975).…”

Operations analysis of behavioral observation procedures: a taxonomy for modeling in an expert training system

“…This article also provides operational specifications that would allow us to extend an expert software system, previously described in this journal (J. M. Ray & Ray, 2008), that models only two of the existing procedures reported in the behavior observation research literature. This system, called train-to-code 2 (TTC), is a comprehensive program for training behavioral observation/recording repertories using errorless training strategies (Terrace, 1963).…”

“…In TTC, Continuous Coding provides a frame-by-frame classification of any incorporated video source and is thus sufficient to provide feedback and correction for any trainee learning to apply these codes (e.g., J. M. Ray & Ray, 2008). However, not all continuous observational procedures involve TTC's explicit coding using an exhaustive set of behavioral categories, thus resulting in continuous observation, but without a continuous record of all behaviors occurring during the entire time period.…”