Developing a patient-centered outcome measure for complementary and alternative medicine therapies II: Refining content validity through cognitive interviews

Abstract

BackgroundAvailable measures of patient-reported outcomes for complementary and alternative medicine (CAM) inadequately capture the range of patient-reported treatment effects. The Self-Assessment of Change questionnaire was developed to measure multi-dimensional shifts in well-being for CAM users. With content derived from patient narratives, items were subsequently focused through interviews on a new cohort of participants. Here we present the development of the final version in which the content and format is refined through cognitive interviews.MethodsWe conducted cognitive interviews across five iterations of questionnaire refinement with a culturally diverse sample of 28 CAM users. In each iteration, participant critiques were used to revise the questionnaire, which was then re-tested in subsequent rounds of cognitive interviews. Following all five iterations, transcripts of cognitive interviews were systematically coded and analyzed to examine participants' understanding of the format and content of the final questionnaire. Based on this data, we established summary descriptions and selected exemplar quotations for each word pair on the final questionnaire.ResultsThe final version of the Self-Assessment of Change questionnaire (SAC) includes 16 word pairs, nine of which remained unchanged from the original draft. Participants consistently said that these stable word pairs represented opposite ends of the same domain of experience and the meanings of these terms were stable across the participant pool. Five pairs underwent revision and two word pairs were added. Four word pairs were eliminated for redundancy or because participants did not agree on the meaning of the terms. Cognitive interviews indicate that participants understood the format of the questionnaire and considered each word pair to represent opposite poles of a shared domain of experience.ConclusionsWe have placed lay language and direct experience at the center of questionnaire revision and refinement. In so doing, we provide an innovative model for the development of truly patient-centered outcome measures. Although this instrument was designed and tested in a CAM-specific population, it may be useful in assessing multi-dimensional shifts in well-being across a broader patient population.

Group Outdoor Health Walks Using Activity Trackers: Measurement and Implementation Insight from a Mixed Methods Feasibility Study

“…The field of integrative medicine has faced this challenge and developed the Self-Assessment of Change (SAC) scale, a holistic measure of biopsychosocial-spiritual health. The SAC scale is sensitive to changes experienced by people engaged in complex interventions to promote health and wellbeing [54][55][56], thus it could be applicable for assessment of the biopsychosocial-spiritual health outcomes in NBI studies.…”

“…The SAC scale has been administered to more than 600 individuals across six different studies, including RCTs and programme evaluations [55,56]. Participants have used the full 100 mm line and have shown a willingness to indicate that domains were worsening, as well as improving.…”

Cognitive interviews guide design of a new CAM patient expectations questionnaire

“…Cognitive interview participants are asked to verbally articulate their thought processes related to selection of responses during or immediately after answering specific questionnaire items [17,21,23]. There are two distinct cognitive interviewing paradigms [22,25].…”

Is There a Role for Complementary and Alternative Medicine in Preventive and Promotive Health? An Anthropological Assessment in the Context of U.S. Health Reform

“…For example, a Self‐Assessment of Change Questionnaire was developed by Ritenbaugh and colleagues after analyzing patients’ narratives of first‐hand experiences with CAM therapies to identify items grounded in lay language that identified multidimensional shifts in well‐being not captured by existing instruments (Ritenbaugh et al ). This instrument has been extensively evaluated to confirm that it is representative of patients’ lived‐experiences with CAM (Thompson et al ). This is just one example of the type of research anthropologists may be involved in that can contribute to better evaluation of the effectiveness of interventions from an experience‐near perspective.…”