The analysis and interpretation of cognitive interviews for instrument development

Abstract

Cognitive interviews assess respondents' understanding of questionnaire items and are increasingly used to improve instrument design. Although investigators have described the contributions of cognitive interviews for instrument development, few guidelines are available for analyzing data from cognitive interviews when they are used for that purpose. In this article we address the development and application of analytic strategies for summarizing, interpreting, and using data from cognitive interviews that were conducted during the process of creating a measure of parental management of childhood chronic conditions. We discuss the contribution of cognitive interviews to establishing content validity and address the importance of developing standardized guidelines for analyzing and interpreting cognitive interviews in order to maximize their usefulness for instrument development.

Cancer Family Caregivers: A New Direction for Interventions

“…The quantitative survey results are described elsewhere. 14,15 In order to increase the rigor of our instrument development, 16 we also planned on conducting in-depth interviews on a subsample of these same family caregivers. However, it soon became clear that participants wanted to share their stories of caregiving, so we invited all caregivers to be interviewed.…”

Eliciting the impact of digital consulting for young people living with long term conditions (LYNC study): which Patient Reported Outcome Measure? (Preprint)

“…We undertook a qualitative exploration of the PHBQ and the PAM using cognitive interviewing, an established method for the development and revision of PROMs (31)(32)(33)(34)(35).…”

Adolescent Reactions to Icon-Driven Response Modes in a Tablet-Based Health Screening Tool

“…Prior to coding the transcripts, the required action and the intent or meaning of the icon or gestural actions was specified for each mode and each question, to ensure consistency between the primary and secondary coder; as well, a set of a priori codes were developed 16,19 to flag whether participants struggled to understand how to use the icon or gesture or not, whether their vocalized understanding of the question prompt, the icon meaning, and the response options fit the intended meaning. These were coded in an itemby-item review within interviews 18 to assess comprehension, but also coded across questions with similar response modes, to determine understanding of the action required with the response mode. Additional inductive coding of emotive responses, value statements as well as potential variations across demographic characteristics, such as participant age, gender, and English as a second language (ESL) status, were developed during the reading and rereading of the transcripts and field notes across individuals and across item areas, to identify inconsistency or commonality of responses and flag potential issues of acceptability in addition to clarity.…”

“…And 3) Do diverse adolescents express predominantly positive or negative views about, or a preference for completing health screens that involve electronic tablets and interactive response options? interviews has been used for more than 30 years 17 as one of the primary methods of assessing survey item clarity, acceptability, and comprehension across a diverse range of age groups 18 . It has also been recommended widely for usability studies in computer software and internet use studies 19 , including for interactive e-health software use 16,19 .…”