Detecting Scale Recalibration in Survey Research

Abstract

The accurate detection of scale recalibration is an issue that should not be ignored by organizational researchers and practitioners concerned with the accurate assessment of change interventions. In an effort to more effectively operationalize one methodology that is designed to detect scale recalibration (using ideal scales), three research questions were developed and investigated in a laboratory study utilizing videotape technology. It was concluded that ideal scale methodology can be used to accurately detect the presence of scale recalibration in survey research data. The implications of these findings for both researchers and practitioners are discussed.Scale recalibration, also referred to as beta change (Golembiewski, Billingsley, & Yeager, 1976), occurs in temporal survey research when respondents (1) give different ratings across time for identical behavior or (2) give the same rating across time for different behaviors. In the first case, identical behavior at Time 1 and Time 2 may be rated a 2 and a 3, respectively. That is, a researcher may conclude that behavior has changed (based on the changes in the ratings) when, in fact, behavior has remained unchanged. In the second case, different behaviors at Time 1 and Time 2 may be rated

Context and Prior Impression Effects on Attention, Judgment Standards, and Ratings: Contrast Effects Revisited

“…Our conclusion about the stability of standards of evaluation differs from that of Maurer and Alexander (1991) with respect to context, and from that of Buckley and Armenakis (1987) with respect to time. Thus, whereas the evaluative level of the ideal remains high regardless of the context, some components of this construct are context-dependent.…”

“…As Barsalou (1985) has shown, and as Buckley and Armenakis (1987) discuss, categories constructed (or retrieved) in the service of goals are represented by ideals. Rather than containing average or typical attribute values, such as might be found with common taxonomic categories (e.g., birds), ideals contain extremes that are relevant to the goal in question.…”

“…Buckley and Armenakis' (1987) subjects rated only ideal performances on standardized scales. Maurer and Alexander (1 99 1) had subjects rate ideal, typical, and worst possible performers on standardized rating scales.…”

“…Maurer and Alexander (1 99 1) had subjects rate ideal, typical, and worst possible performers on standardized rating scales. Barsalou (1989, using free-response ideal elicitation, found ideals to be unstable over time, while Buckley and Armenakis (1987) found stability once the ideal had been formed. The former found no influences of context performances on the evaluative level of the ideal, while the latter did.…”

“…Two possibilities exist. The first argument is that exposure to evaluatively polarized context performances changes the implicit scaling function of behaviors onto an evaluative dimension by moving the scale anchors up or down (Buckley & Armenakis, 1987;Latham, Wexley, & Pursell, 1975). Second, rather than changing the rater's overall standard of goodness of a performance, context performances might serve to make some dimensions of standards more or less accessible, changing the qualitative nature of the standard.…”