When data are collected via anonymous Internet surveys, particularly under conditions of obligatory participation (such as with student samples), data quality can be a concern. However, little guidance exists in the published literature regarding techniques for detecting careless responses. Previously several potential approaches have been suggested for identifying careless respondents via indices computed from the data, yet almost no prior work has examined the relationships among these indicators or the types of data patterns identified by each. In 2 studies, we examined several methods for identifying careless responses, including (a) special items designed to detect careless response, (b) response consistency indices formed from responses to typical survey items, (c) multivariate outlier analysis, (d) response time, and (e) self-reported diligence. Results indicated that there are two distinct patterns of careless response (random and nonrandom) and that different indices are needed to identify these different response patterns. We also found that approximately 10%-12% of undergraduates completing a lengthy survey for course credit were identified as careless responders. In Study 2, we simulated data with known random response patterns to determine the efficacy of several indicators of careless response. We found that the nature of the data strongly influenced the efficacy of the indices to identify careless responses. Recommendations include using identified rather than anonymous responses, incorporating instructed response items before data collection, as well as computing consistency indices and multivariate outlier analysis to ensure high-quality data.

No abstract

This article concerns the real-world importance of leadership for the success or failure of organizations and social institutions. The authors propose conceptualizing leadership and evaluating leaders in terms of the performance of the team or organization for which they are responsible. The authors next offer a taxonomy of the dependent variables used as criteria in leadership studies. A review of research using this taxonomy suggests that the vast empirical literature on leadership may tell us more about the success of individual managerial careers than the success of these people in leading groups, teams, and organizations. The authors then summarize the evidence showing that leaders do indeed affect the performance of organizations--for better or for worse--and conclude by describing the mechanisms through which they do so.

No abstract

The purpose of this study was to test whether a multisource performance appraisal instrument exhibited measurement invariance across different groups of raters. Multiple-groups confirmatory factor analysis as well as item response theory (IRT) techniques were used to test for invariance of the rating instrument across self, peer, supervisor, and subordinate raters. The results of the confirmatory factor analysis indicated that the rating instrument was invariant across these rater groups. The IRT analysis yielded some evidence of differential item and test functioning, but it was limited to the effects of just 3 items and was trivial in magnitude. Taken together, the results suggest that the rating instrument could be regarded as invariant across the rater groups, thus supporting the practice of directly comparing their ratings. Implications for research and practice are discussed, as well as for understanding the meaning of between-source rating discrepancies.

Despite keen interest, questions remain about defining and measuring the behavioral flexibility of managers. This paper reports a conceptual and empirical comparison of three alternative methods of assessing this construct. Results suggest that the way managerial flexibility is typically assessed in practice -as a trait-like characteristic with coworker ratings that describe a general tendency to vary behavior across situations -is deficient. However, more complex models that represent flexibility as a higher-order construct reflecting mastery of specific and opposing behaviors in both the social/ interpersonal domain and the functional/organizational domain show promise. They demonstrate construct validity evidence, predict as much as 42% of the variance in overall effectiveness, and provide more specific diagnostic information to guide behavior change.

Kaiser, Craig, Overfield, & Yarborough, this issue). However, there is a noteworthy gap in this otherwise extensive literature. Because job requirements vary with organizational level, it seems logical to suppose that the behaviors that constitute effective performance also differ across levels. But, as Zaccaro (2001) andYukl (2006) have pointed out, the published literature contains no direct empirical tests of the proposition that the behaviors that predict managerial effectiveness vary with organizational level. Nor is there direct evidence for the assumed discontinuities in these differences-where the same behavior is positively related to effectiveness at one level but negatively related to effectiveness at another level (cf. Freedman, 1998; Kaiser et al., this issue).This article reports a direct test of the assumption that the behaviors that predict effectiveness are different for jobs at the bottom, middle, and top. This study is the first we know of to use an identical set of measures across a range of organizations to directly test whether the behaviors that predict managerial effectiveness change with organizational level. Such an empirical test is important not only for theoretical reasons but also because of the widespread application of such assumptions in practice on the basis of popular books lacking an empirical basis, including The Leadership Pipeline (Charan, Drotter, & Noel, 2001) and What Got You Here Won't Get You There (Goldsmith & Reiter, 2007). Is the popular fascination with the Leadership Pipeline model and related best practices warranted by data, or is this just another fad lacking empirical support?

It has become popular in the practices of leadership development and talent management to segment managers at different organizational levels in order to focus on the unique requirements thought to characterize jobs at each level. This movement has been spurred by popular books that emphasize differences in the nature of managerial work at different hierarchical levels. Seemingly independent of popular work in this area has been scientific research to describe differences in managerial jobs across organizational levels. The present article summarizes the extensive research literature on level differences in managerial jobs in terms of three broad generalizations: The number of distinct management levels, five different ways to