The Effects of Order of Stimulus Presentation on Ratings of Counseling Performance

Abstract

This study was designed to examine the effects of order of stimulus presentation on observer ratings of counseling performance. Two videotaped counseling simulations, determined by judges to represent either high or low quality performance, served as the stimuli. Participants were randomly assigned to one of two levels of the order effect: Group 1 viewed the videotape with low quality performance first, followed by the high quality performance videotape; Group 2 viewed the high quality performance videotape first, followed by the low quality performance videotape. Four widely used measures of counseling performance served as dependent measures. Results revealed a statistically significant interaction between quality of performance and the order in which the performances were rated.The assessment of counseling performance plays an important role in both training and research. Assessment is central to the evaluation of student performance and progress. Accurate feedback, which is an essential element of effective training methods (Ford, 1979), is highly reliant on accurate assessment. The effectiveness of counselor training techniques is determined through assessment as well. One of the most widely used methods for assessing counseling performance is the rating scale (Tinsley & Weiss, 1975), which typically consists of items developed around specific characteristics of performance and presented on a multipoint scale. Observers are required to make subjective judgments regarding the degree to which each characteristic is present in a given Jody L.

Effects of order of presentation on perceptions of the counselor.

“…The current study extends our earlier study (Newman & Fuqua, 1986) in two ways. The earlier study involved simulated counseling interviews designed to vary only in the quality of counseling performance.…”

“…In a previous study (Newman and Fuqua, 1986), we directly investigated the effects of order of stimulus presentation on ratings of counseling performance. Subjects in that study were undergraduate students who were randomly assigned to two groups.…”

“…Group 2 also rated both performances, but in reversed order. Results indicated that “when the low quality condition was presented first, the magnitude of ratings of high quality performance increased significantly” (Newman & Fuqua, 1986, p. 13). Differences between ratings of low-quality performance viewed first and ratings of low-quality performance viewed second were in the expected direction but were not statistically significant.…”

Bias in participant ratings of psychotherapy process: An initial generalizability study.

“…It may be that therapists are rated more favorably in some orders than others (a Therapist × Order interaction). This can occur because of comparison effects—for example, a therapist who is rated favorably overall may be rated more favorably when a less competent therapist is viewed first (Newman & Fuqua, 1986). It can also be that the average rating (across all therapists) is higher in some orders than others (an order main effect), perhaps because of anchoring and adjustment effects (Tversky & Kahneman, 1974).…”

The Dynamics and Validity of the Group Selection Interview

“…Hypothesis 2: The order in which the applicant in the group interview format is asked to respond to the question will affect the quality and originality of each response (Friedrich & Smith, 1998;Haradlsson & Gissurarson, 1985;Kerstholt & Jackson, 1998;Newman & Fuqua, 1986).…”