Common Method Variance in IS Research: A Comparison of Alternative Approaches and a Reanalysis of Past Research

Abstract

Despite recurring concerns about common method variance (CMV) in survey research, the information systems (IS) community remains largely uncertain of the extent of such potential biases. To address this uncertainty, this paper attempts to systematically examine the impact of CMV on the inferences drawn from survey research in the IS area. First, we describe the available approaches for assessing CMV and conduct an empirical study to compare them. From an actual survey involving 227 respondents, we find that although CMV is present in the research areas examined, such biases are not substantial. The results also suggest that few differences exist between the relatively new marker-variable technique and other well-established conventional tools in terms of their ability to detect CMV. Accordingly, the marker-variable technique was employed to infer the effect of CMV on correlations from previously published studies. Our findings, based on the reanalysis of 216 correlations, suggest that the inflated correlation caused by CMV may be expected to be on the order of 0.10 or less, and most of the originally significant correlations remain significant even after controlling for CMV. Finally, by extending the marker-variable technique, we examined the effect of CMV on structural relationships in past literature. Our reanalysis reveals that contrary to the concerns of some skeptics, CMV-adjusted structural relationships not only remain largely significant but also are not statistically differentiable from uncorrected estimates. In summary, this comprehensive and systematic analysis offers initial evidence that (1) the marker-variable technique can serve as a convenient, yet effective, tool for accounting for CMV, and (2) common method biases in the IS domain are not as serious as those found in other disciplines.common method variance, method biases, marker variable, logit analysis, path analysis

From the Editors: Common method variance in international business research

“…Lindell and Whitney (2001), Podsakoff et al (2003) and Malhotra, Kim, and Patil (2006) review several statistical methods that are more sophisticated than Harman's test, which can be used to test and possibly control for CMV. Different statistical remedies are available for different types of research settings and different sources of CMV.…”

“…7 Note that a complete identification of all sources of CMV is not necessary. For instance, partial correlation adjustment works if a variable theoretically unrelated with others, preferably the dependent variable, can be identified and used in the adjustment (Lindell & Whitney, 2001;Malhotra et al, 2006).…”

Are public officials really less satisfied than private sector workers?A comparative study in Brazil

“…Usually, self-reports questionnaires (because self-report measures are supposed to be correlated), social desirability (it might inflate observed correlations), negative affectivity (respondent's negative emotions might bias the relationships in a negative direction), and acquiescence (the tendency to agree with the items no matter the content) are cited as most usual sources of common method variance. As result, several statistical procedures were developed to overcome this problem, such as the traditional MTMM matrix procedure, the CFA-based MTMM technique, Harman's single-factor test, and marker-variable technique (Malhotra, Kim and Patil, 2006).…”

“Trust but verify”? The performance implications of verification strategies in trusting relationships

“…First, we compared a marker variable, unrelated to the variables in this study ("How involved are you in your firm's dealings with this supplier? "), and found it uncorrelated with all constructs in this study (Lindell and Whitney 2001;Malhotra et al 2006). We also used a confirmatory factor analysis approach to Harmon's one-factor test (Podsakoff et al 2003;Sanchez and Brock 1996).…”