Development of a short‐form version of the Reading the Mind in the Eyes Test for assessing theory of mind in older adults

Abstract

Background: The Reading the Mind in the Eyes test (RMET) is a 36-item assessment for theory of mind (ToM) performance. While this measure has been shown to be sensitive to age-related ToM difficulties, there are no established cutoffs or guidelines currently available that are specific to older adults. This article seeks to validate a short-form version of the RMET appropriate for use in such populations. Methods: Cross-sectional data from 295 participants (mean age 86 years) from the Sydney Memory and Ageing Study, a longitudinal community observational cohort. Participants underwent an assessment battery that included the RMET. Individuals who scored >1SD below the RMET scores of cognitively normal participants were deemed to have below average RMET scores. Various model-building methods were used to generate short-form solutions of the RMET, which were compared with previously validated versions in their predictive power for below average full RMET performance. Results: Individuals with below average RMET performance tended to be older and have poorer global cognition. Of the eight short-form solutions, the 21-item version generated using genetic algorithm exhibited the best classification performance with an area under the receiver operating curve (AUROC) of 0.98 and had 93.2% accuracy in classifying individuals with below average ToM. A shorter 10-item solution derived by ant colony optimization also had acceptable performance. Conclusion: We recommend the 21-item version of the RMET for use in older adult populations for identifying individuals with impaired ToM. Where an even shorter version is needed with a trade-off of slightly reduced performance, the 10-item version is acceptable. K E Y W O R D S neurocognitive disorders, Reading the Mind in the Eyes Test, social cognition, theory of mind 1 | INTRODUCTION Social cognition refers to the skills and abilities that humans use to process social information, thereby allowing them to navigate and interpret social scenarios, and behave in a socially appropriate manner. 1 Such skills include the ability to infer the cognitive and emotional states of others through social cues, such as facial expressions, and the ability to respond to these cues in a socially acceptable manner. 2 Social cognitive impairment is now recognized to be a common and often debilitating feature of many clinical disorders, including

Confirmatory factor analysis of the Brief Reading the Mind in the Eyes Test (RMET-10) in three large samples reveals strong evidence of unidimensionality

“…Thus, the aim of the present study was to present further validity evidence on the RMET-10 in three large, independent datasets. Across three samples (total N = 3,121), we found that the RMET-10's hypothesized unidimensional model provided an excellent fit to the data (see Table 2), corroborating the favorable conclusions reached by Olderbak et al (2015) and Chander et al (2020) regarding the RMET-10's factorial validity. Despite observing strong factorial validity evidence, however, the reliability estimates obtained in the present study fell somewhat short of conventionally cited guidelines (e.g., .70 or .80).…”

“…Like the original validation studies of the RMET-10 (Olderbak et al, 2015), all data were solely collected from convenience samples of United States residents. As such, between our own results and those of Olderbak et al (2015) and Chander et al's (2020) study of Australian adults, we are limited in what can be concluded regarding the RMET-10's validity in other cultural contexts (e.g., non-Western contexts where emotion categories, vocabulary or display norms may differ, among many other factors). A clear priority for future research is thus to test the RMET-10 in other contexts.…”

“…Interestingly, Sample 2 demonstrated slightly higher omega estimates compared to Sample 1 and 3 (consistent with the higher observed inter-item correlations and factor loadings, perhaps in part attributable to Sample 2 being the only one that completed just the 10 items in the RMET-10). Omega values across all three samples were lower than Olderbak et al's (2015) reported omega of .73, but uniformly higher than alpha estimates reported in Chander et al (2020;alpha = 0.47).…”

“…Among the most promising of these efforts has been the RMET-10, developed by Olderbak et al (2015). To our knowledge, Chander et al (2020) is the only independent factor analytic study of the RMET-10 thus far, finding that the RMET-10 was sufficiently reliable and indeed comparable to the full-length version (and in fact, the highest performing amongst its short-version peers). However, Chander et al's study relied on just a single sample of 295 participants.…”

“…In light of its extensive usage as an operationalization of cognitive empathy, particularly in clinical settings (e.g., Chander et al, 2020), the psychometric properties of the RMET have rightly received a great deal of attention. Despite the RMET's widespread use, however, the extant psychometric literature indicates that the full 36-item Eyes Test has poor internal consistency and test-retest reliability, alongside poor factorial validity and convergent validity (Harkness et al, 2010;Olderbak et al, 2015;Voracek & Dressler, 2006;Higgins, Ross, Langdon, & Polito, 2022).…”