Source-Monitoring Training: Toward Reducing Rater Expectancy Effects in Behavioral Measurement.

Abstract

The authors developed a source-monitoring procedure to reduce the biasing effects of rater expectations on behavioral measurement. Study participants (N = 224) were given positive or negative information regarding the performance of a group and, after observing the group, were assigned to a source-monitoring or control condition. Raters in the source-monitoring condition were instructed to report only behaviors that evoked detailed memories (remember judgments) and to avoid reporting behaviors based on feelings of familiarity (know judgments). Results revealed that controlling raters' response strategy reduced (and often eliminated) the biasing effects of performance expectations. These findings advance our understanding of the performance-cue bias and offer a potentially useful technique for decreasing rater bias.

Union leadership and member attitudes: A multi-level analysis.

“…Some of the scale items were more general, however, such as counseling members about work related problems, and therefore more likely to access semantic memory. With respect to scale items triggering vivid memory as opposed to a general feeling (Martell & Evans, 2005), we believe that the union context is sufficiently different from the teachers' regular work context-they are exposed to the union context less regularly than to their classrooms and school buildings and may therefore have had a better recall of what they actually experienced or observed instead of a more general impression of internal leadership. We cannot argue that our measure is free from implicit theory bias, however; we can argue only that we did our best to minimize it.…”

“…There are no scale construction procedures that will eliminate them, but there are steps one can take to encourage the raters' use of episodic memory (remembering) instead of semantic memory (knowing; see Martell & Evans, 2005). One can ground measurement in specific events in specific contexts that are less influenced by the raters' general knowledge about leadership or leader traits and prototypes (Gioia & Sims, 1985;Uleman, 1991).…”

Reconsidering the accuracy of follower leadership ratings

“…In this sense, false alarms affect discrimination as well as bias. Martell and Evans (2005) suggest that reliance on prototypical leader behaviors not only produces false memories, but also fosters heightened feelings of familiarity with prototypical leader behaviors, which in turn causes raters to adopt more liberal decision criteria.…”

“…Rater training might improve accuracy in follower ratings of leader behavior. For example, Martell and Evans (2005) trained raters to report only behaviors that evoked detailed memories and to avoid behaviors based on feelings of familiarity.…”

“…In this sense, false alarms affect discrimination as well as bias. Martell and Evans (2005) suggest that reliance on prototypical leader behaviors not only produces false memories, but also fosters heightened feelings of familiarity with prototypical leader behaviors, which in turn causes raters to adopt more liberal decision criteria.Similarly, followers' affective reactions to leaders may systematically influence ratings of leader behavior (Bono & Ilies, 2006;Naidoo, Kohari, Lord, & DuBois, 2010) by impacting both memory sensitivity (Allen, Kaut, & Lord, 2008) and bias, as positive affect engenders more liberal thresholds.Taken together, the evidence suggests that implicit leadership theories and other aspects of raters such as their affective reactions present a significant challenge to ensuring accuracy in follower ratings of leader behavior. Thus, their effects on the data used for theory testing or leadership development represents a critical issue for the leadership field.…”