The Effect of Response Style on Self-Reported Conscientiousness Across 20 Countries

Abstract

Rankings of countries on mean levels of self-reported Conscientiousness continue to puzzle researchers. Based on the hypothesis that cross-cultural differences in the tendency to prefer extreme response categories of ordinal rating scales over moderate categories can influence the comparability of self-reports, this study investigated possible effects of response style on the mean levels of self-reported Conscientiousness in 22 samples from 20 countries. Extreme and neutral responding were estimated based on respondents' ratings of 30 hypothetical people described in short vignettes. In the vignette ratings, clear cross-sample differences in extreme and neutral responding emerged. These responding style differences were correlated with mean self-reported Conscientiousness scores. Correcting self-reports for extreme and neutral responding changed sample rankings of Conscientiousness, as well as the predictive validities of these rankings for external criteria. The findings suggest that the puzzling country rankings of self-reported Conscientiousness may to some extent result from differences in response styles.

Acculturation of Personality

“…The co-existence of contradictory emotions in the Anxiety scale might have affected its coherence negatively, leading to a low alpha for Neuroticism. The low reliability for Conscientiousness in Japanese, on the other hand, is consistent with the cross-cultural studies reporting low relevance of this dimension to the Japanese culture (e.g., Mõttus et al, in press). Overall, the similarity of Japanese Americans to European Americans in higher reliabilities for Neuroticism and Conscientiousness is consistent with the acculturation of personality patterns towards the new culture and away from the heritage culture.…”

On Enhancing the Cross–Cultural Comparability of Likert–Scale Personality and Value Measures: A Comparison of Common Procedures

“…When the original responses in this study were adjusted by means of the vignettes, the country orderings changed little and the correlations with external criteria moved generally in the expected direction, but remained weak. A somewhat larger shift of country‐level scores in the expected direction was achieved when vignette scores were used to derive response‐style indices (Mõttus et al, ), and raw responses were subsequently corrected by means of these indices. Given the relevance of conscientiousness for some of the most counter‐intuitive findings in country‐level personality scores, the present study also examines reference‐group effects in trait conscientiousness, but further explores reference‐group effects in vignettes of values and response styles (as described in the Measures section) and examines these effects for the scores on all of the Big Five factors and Schwartz's values (Schwartz & Sagiv, ).…”

“…The effectiveness of these procedures is evaluated with five types of evidence: (i) for the anchoring vignettes, the equivalence of the vignettes; (ii) the internal consistency of the factor or scale scores from the different procedures (i.e. Cronbach's alpha), which serves as a prerequisite psychometric check before conducting invariance tests; (iii) the effects of the different procedures on equivalence (as a baseline, we expect that each procedure would lead to some improvement of comparability over the raw continuous scores, given that all the procedures we test have been proposed to alleviate bias issues); (iv) the effects on the nomological networks of the self‐assessed personality and value scores; and (v) the effects on the external validity of country‐level conscientiousness scores, which we operationalize by means of correlations with country‐level development indices in line with previous research (Mõttus et al, ; Mõttus et al, ).…”

The inaccuracy of national character stereotypes

“…Cultures also differ in self-enhancement; that might bias self-report data, but should not affect informant ratings of personality. Mõttus and colleagues (Mõttus, Allik, Realo, Rossier, et al, 2012) showed that extreme responding, although it had little effect on individual scores, had a larger effect on culture-level scores of Conscientiousness. Nevertheless, the rank-order of cultures was similar when scores corrected for extreme responding were compared to uncorrected scores, rho = .68, p < .001.…”