Validation of Subjective Well-Being Measures Using Item Response Theory

Abstract

Background: Subjective well-being refers to the extent to which a person believes or feels that her life is going well. It is considered as one of the best available proxies for a broader, more canonical form of well-being. For over 30 years, one important distinction in the conceptualization of subjective well-being is the contrast between more affective evaluations of biological emotional reactions and more cognitive evaluations of one's life in relation to a psychologically self-imposed ideal. More recently, researchers have suggested the addition of harmony in life, comprising behavioral evaluations of how one is doing in a social context. Since measures used to assess subjective well-being are self-reports, often validated only using Classical Test Theory, our aim was to focus on the psychometric properties of the measures using Item Response Theory. Method: A total of 1000 participants responded to the Positive Affect Negative Affect Schedule. At random, half of the participants answered to the Satisfaction with Life Scale or to the Harmony in life Scale. First, we evaluate and provide enough evidence of unidimensionality for each scale. Next, we conducted graded response models to validate the psychometric properties of the subjective well-being scales. Results: All scales showed varied frequency item distribution, high discrimination values (Alphas), and had different difficulty parameters (Beta) on each response options. For example, we identified items that respondents found difficult to endorse at the highest/lowest point of the scales (e.g., "Proud" for positive affect; item 5, "If I could live my life over, I would change almost nothing," for life satisfaction; and item 3, "I am in harmony," for harmony in life). In addition, all scales could cover a good portion of the range of subjective well-being (Theta): −2.50 to 2.30 for positive affect, −1.00 to 3.50 for negative affect, −2.40 to 2.50 for life satisfaction, and −2.40 to 2.50 for harmony in life. Importantly, for all scales, there were weak reliability for respondents with extreme latent scores of subjective well-being. Conclusion: The affective component, especially low levels of negative affect, were less accurately measured, while both the cognitive and social component were covered to an equal degree. There was less reliability for respondents with extreme latent scores

Peer Review #2 of "Validation of a general subjective well-being factor using Classical Test Theory (v0.2)"

“…Moreover, in the bifactor model, 74% of the variance for Alert" and 58% of the variance for "Attentive" were error variances (uniqueness) and did not share reliable variance with any other of items within the model. So, we recommend to remove or replace these items following the recommendations found elsewhere using Item Response Theory (e.g., Nima, Cloninger, Persson, Sikström & Garcia, 2020). The HILS items contributed to a very high degree to the general factor (SWBS), but they did not contribute to HILS itself.…”

“…In this circumplex, "Alert" and "Attentive" are categorized on the highest point of the high activation dimension and at the lowest point of the positive dimension. It is therefore plausible to suggest that it is this specific feature (i.e., low positive and high activation types of emotions) in these items what causes the problems highlighted using both CTT here and Item Response Theory in other studies (e.g., Nima et al, 2020).…”

Peer Review #1 of "Validation of a general subjective well-being factor using Classical Test Theory (v0.1)"

“…Response Theory was applied using a different sample (Nima, Cloninger, Persson, Sikström & Garcia, 2020). Regarding affect, some theories suggest that besides the dimension of positive and negative, there is a high and low activation dimension (Russell, 1980;Russell & Feldman Barret, 1999).…”

“…One of the limitations with factorial models is that they are heavily influenced by the sample characteristics, such as, number of participants (e.g., Nima et al, 2020). However, according to Thompson (2004) if the factors are defined by four or more measured variables with structure coefficients < .60, then the sample size if not important.…”

Psychometric properties of the Multi-group Ethnic Identity Measure (MEIM) in a sample of Iranian young adults

“…The cognitive component of subjective well-being refers to an assessment of the quality of different aspects of one’s life and a general feeling of satisfaction with life ( Gilman, Huebner & Laughlin, 2000 ). Life satisfaction, positive affect, and negative affect are considered as independent phenomena ( Lucas, Diener & Suh, 1996 ) and are therefore often evaluated individually rather than as an integrated entity (for other perspectives see Nima et al., 2020a ; Nima et al., 2020b ; Garcia et al., in press ).…”