Cronbach’s Alpha under Insufficient Effort Responding: An Analytic Approach

Abstract

Surveys commonly suffer from insufficient effort responding (IER). If not accounted for, IER can cause biases and lead to false conclusions. In particular, Cronbach's alpha has been empirically observed to either deflate or inflate due to IER. This paper will elucidate how IER impacts Cronbach's alpha in a variety of situations. Previous results concerning internal consistency under mixture models are extended to obtain a characterization of Cronbach's alpha in terms of item validities, average variances, and average covariances. The characterization is then applied to contaminating distributions representing various types of IER. The discussion will provide commentary on previous simulation-based investigations, confirming some previous hypotheses for the common types of IER, but also revealing possibilities from newly considered responding patterns. Specifically, it is possible that the bias can change from negative to positive (and vice versa) as the proportion of contamination increases.

Validation of the Multidimensional Assessment of Interoceptive Awareness (MAIA-2) questionnaire in hospitalized patients with major depressive disorder

“…A comparable pattern has been reported in another longitudinal study recruiting depressed patients [ 32 ]. In terms of theoretical mechanisms, it is possible that there is an interplay between psychopathological symptomology and the properties of internal consistency reliability: it has been shown that reliability of a self-report Likert scale may be affected by “insufficient effort responding” (careless responding styles), for example by “straightlining” identical responses or giving random ratings [ 37 ], resulting in biased estimates of internal consistency [ 38 ]. Accordingly, recent research has shown that psychiatric symptoms are associated with less consistent response patterns on self-rated scales [ 39 ].…”

Principals' Attitudes, Knowledge, and Beliefs Regarding Active Learning Methodologies in Spain

“…Once the validation process was completed, the reliability of the questionnaire was analyzed. For this, a global statistical analysis and an individual item analysis for each item were carried out using Cronbach's alpha; once these analyses were completed, the Spearman-Brown and Guttmann coefficients were used as complementary tests [59].…”

Women entrepreneurial intentions in subsistence marketplaces: The role of entrepreneurial orientation and demographic profiles in Zimbabwe

“…With a view to validating the research constructs prior to structural equation modeling, Carden et al (2019) recommend the need to conduct validity testing of the extracted dimensions. For the construct validity, the researcher considered the use of the Confirmatory Factor Analysis (CFA) as prescribed by Tabachnick and Fidell (2007) and affirmed by Hair et al (2018).…”

“…Moreso, according to Field (2016) together with Carden et al (2019), the general standard threshold for the assessment of reliability is 0.7, however, alpha loadings as low as 0.6 are still considered to be reliable while those above 0.7 are considered to be the most desirable (Sweet & Grace-Martin, 2012). From the above results, the least observed alpha statistic was 0.350 for the sixth component and this was less than 0.70, while the second least was the fifth component (0.497), again, less than 0.70.…”