Factor structure and measurement invariance of the Health Education Impact Questionnaire: Does the subjectivity of the response perspective threaten the contextual validity of inferences?

Abstract

Objective:On-going evidence is required to support the validity of inferences about change and group differences in the evaluation of health programs, particularly when self-report scales requiring substantial subjectivity in response generation are used as outcome measures. Following this reasoning, the aim of this study was to replicate the factor structure and investigate the measurement invariance of the latest version of the Health Education Impact Questionnaire, a widely used health program evaluation measure.Methods:An archived dataset of responses to the most recent version of the English-language Health Education Impact Questionnaire that uses four rather than six response options (N = 3221) was analysed using exploratory structural equation modelling and confirmatory factor analysis appropriate for ordered categorical data. Metric and scalar invariance were studied following recent recommendations in the literature to apply fully invariant unconditional models with minimum constraints necessary for model identification.Results:The original eight-factor structure was replicated and all but one of the scales (Self Monitoring and Insight) was found to consist of unifactorial items with reliability of ⩾0.8 and satisfactory discriminant validity. Configural, metric and scalar invariance were established across pre-test to post-test and population sub-groups (sex, age, education, ethnic background).Conclusion:The results support the high level of interest in the Health Education Impact Questionnaire, particularly for use as a pre-test/post-test measure in experimental studies, other pre–post evaluation designs and system-level monitoring and evaluation.

Making robust decisions about the impact of health education programs: Psychometric evaluation of the Health Education Impact Questionnaire (heiQ) in diverse patient groups in Norway

“…For this scale, the internal consistency was somewhat low and it was found that half of the items had modest loadings, suggesting that this scale's hypothesized concept may not be as cleanly represented by the present items compared to the other heiQ scales. This finding is consistent with Australian findings, which had resulted in some small adjustments to this particular scale between heiQ version 1.0 and 2.0 [11]. However, before substantial changes are made to the current Norwegian version 2.0, it needs to be explored carefully whether other language versions also show potentially problematic items of the domain in question.…”

“…Further details on the development of the heiQ and the validation of the original and later versions can be found elsewhere [9,11].…”

“…The heiQ is generic and appropriate for a wide range of courses and diseases, user friendly and psychometrically robust [9,11]. The final instrument has eight scales, which have been translated and culturally adapted into over 20 languages, and are currently being applied in clinical trials worldwide, including in Norway [12][13][14][15].…”

“…The process of technical translation, cultural adaptation and measurement equivalence of the heiQ version 2.0 into Norwegian followed a standardized protocol [11,17]. The report can be obtained from the first author (AKW).…”

Percentile ranks and benchmark estimates of change for the Health Education Impact Questionnaire: Normative data from an Australian sample

“…7 Given this finding, it can be concluded that the heiQ items yield equivalent measurement parameters across these critical socio-demographic groups and, when combined into the eight scales, result in unbiased scores that can justifiably be compared across these groups. Relationships between the heiQ scale scores at baseline and these socio-demographic groups are presented in Table 3.…”