On the Folly of Introducing A (Time-Based UMV), While Designing for B (Time-Based CMV)

Abstract

To mitigate concerns of common method variance (CMV), researchers often separate predictor and criterion variables, with temporal separation being the most widely applicable option (e.g., Podsakoff et al., 2012). However, temporal separation is empirically understudied relative to other CMV corrections (Lance et al., 2009;Podsakoff et al., 2012). And despite being recommended, CMV corrections may actually underestimate true correlations by as much as 50% (e.g., Lance et al., 2010;Spector, 2006). In fact, uncommon method variance (UMV) theory (Spector et al., 2019) suggests that temporal separation attenuates observed correlations by way of unshared sources of method variance-such as "Time 1" and "Time 2" data collections (Spector et al., 2019).To investigate UMV in known conditions, I temporally separate items from an established unidimensional Frugality scale (Lastovicka et al., 1999). Weijters et al. (2014) investigated separating this measure psychologically, with each half presented on separate survey pages and using filler scales in-between. Their psychological separation artificially created two factors, or "discriminant validity where there should be none" (Weijters et al., 2014). Here, I split the same unidimensional measure temporally, across Times 1 and 2. Based on UMV theory (Spector et al., 2019), temporal separation should induce two corresponding factors (Hypothesis 1; see Figure 1). MethodWorkers on MTurk-a popular participant source in the social sciences (e.g., Chmielewski & Kucker, 2020)-completed two surveys 1 week apart. Of 685 attentive Time 1 respondents, 486 (71%) completed the second survey and comprised the sample for analysis.The Frugality measure appeared in two 4-item halves on their own page at the beginning of the first survey and end of the second survey. Thirty-nine "filler" items (for an unrelated study