Are cross-cultural comparisons of personality profiles meaningful? Differential item and facet functioning in the Revised NEO Personality Inventory.

Abstract

Measurement invariance is a prerequisite for confident cross-cultural comparisons of personality profiles. Multigroup confirmatory factor analysis was used to detect differential item functioning (DIF) in factor loadings and intercepts for the Revised NEO Personality Inventory (P. T. Costa, Jr., & R. R. McCrae, 1992) in comparisons of college students in the United States (N = 261), Philippines (N = 268), and Mexico (N = 775). About 40%-50% of the items exhibited some form of DIF and item-level noninvariance often carried forward to the facet level at which scores are compared. After excluding DIF items, some facet scales were too short or unreliable for cross-cultural comparisons, and for some other facets, cultural mean differences were reduced or eliminated. The results indicate that considerable caution is warranted in cross-cultural comparisons of personality profiles.

Factorial Invariance Testing under Different Levels of Partial Loading Invariance within a Multiple Group Confirmatory Factor Analysis Model

“…This emphasis is a direct reflection of the essential role that assessment scores play in society (e.g., high-stakes decisions) ranging from education (e.g., teacher evaluations; international student achievement comparisons) to business (e.g., job applicant decisions). In fact, there is an increasing body of evidence that suggests that many observed differences that are cross-cultural may be contaminated by artifacts of measurement or a lack of factorial invariance (Baumgartner & Steenkamp, 2001;Church et al, 2011;Javaras & Ripley, 2007;Poortinga, 1989). These differences may be related to content meaning issues, translation problems, or even response style differences, and in turn, can result in incorrect decisions regarding individuals from different groups as well as group comparisons (French & Finch, 2008b;Millsap & Kwok, 2004;Steinmetz, 2013).…”

“…For example, it seems with various types of scales it is unusual for complete FI to exist (Church et al, 2011;French & Gotch, 2013;French & Mantzicopoulos, 2007). Group equality on some but not all factor parameters is known as partial factorial invariance (PI) and does exist on major instruments such as intelligence measures (Maller & French, 2004).…”

Are the costs of neuroticism inevitable? Evidence of attenuated effects in U.S. Latinas.

“…Neuroticism is similar in non-Latinos in the United States and Latinos in the United States and Mexico (e.g., Benet-Martínez & John, 1998;Church et al, 2011;Ramírez-Esparza, Gosling, Benet-Martínez, Potter, & Pennebaker, 2006). For example, a study by Church et al (2011) of measurement invariance and differential item functioning for self-reported personality in U.S., Mexican, and Philippine samples found that neuroticism is comparable for U.S. and Mexican samples. There is some indication, however, that neuroticism may be viewed more favorably in some sociocultural contexts than others.…”

Cross-Cultural Issues in Personality Assessment

“…Alternatively, Item-Response Theory (IRT)-based methods to examine DIF can be used to establish scalar equivalence (Reise & Henson, 2003), but large sample sizes are required and analyses become more complex when items with Likert scales have to be analyzed. Adopting these methods for comparing sets of personality-descriptive item across cultures learns that many items show DIF, but also that DIF forwards to the facet level, and does not cancel out across multiple items compiling a facet (Church et al, 2011). A second way to demonstrate scalar equivalence is through the use of bilingual retest studies, in which bilingual respondents administer a personality inventory twice.…”

“…A third kind of bias is item bias or differential item functioning (DIF): "DIF occurs when individuals with the same level or amount of a trait, but from different cultural groups, exhibit a different probability of answering the item in the keyed direction" (Church, 2010, p. 154). In a recent study, Church et al (2011) examined DIF in factor loadings and intercepts from a multigroup confirmatory factor analysis (CFA) of NEO PI-R data obtained in the United States, the Philippines, and Mexico, showing that 40%-50% of the items exhibited some form of DIF. Moreover, DIF at the item level also affected the facet level, suggesting that the comparison of mean-level facet and domain scores across cultural groups should be done with caution.…”