Using Qualitative Methods to Assess the Measurement Property of a New HIV Disability Questionnaire

Abstract

The purpose of this article is to describe our experience using a qualitative team approach and predetermined theoretical framework to assess sensibility of a newly developed HIV disability questionnaire. Two interviewers conducted structured qualitative interviews with 22 adults living with HIV, asking participants how well the questionnaire characterized the disability they experienced living with HIV. Data collection and analysis occurred over six stages with four analysts who met throughout. Strengths of our approach included the ability to assess the sensibility of the questionnaire from the perspective of adults living with HIV, collect and analyze data across multiple sites, establish a systematic team analytical process, and enhance rigour through multiple coding, team reflexivity, and interviewer and analyst triangulation. Challenges included increased resources required to coordinate and implement this approach, differential recruitment rates, initial divergent analytical styles, and the potential to miss emerging codes given the structured nature of the analysis. This article offers a methodological process for researchers to use a qualitative team approach with directed content analysis to assess the sensibility of a new health status questionnaire.

Sensibility assessment of the HIV Disability Questionnaire

“…Theoretical saturation, constant comparative analysis, interviewer and analyst triangulation, and validity checks were used to enhance rigor [16][17][18][19][20]. Our approach allowed team members to share overall impressions of the interviews, cross-check the coding of the data, and refine the interview guide and coding scheme [21]. Four researchers independently coded half of the transcripts (multiple coding), and two researchers cross-checked codes for half of the merged transcript data.…”

Questioning the Questionnaire: Methodological Challenges in Measuring Subjective Quality of Life in Nursing Homes Using Cognitive Interviewing Techniques

“…The theme coding system focuses on classic problems inherent in the response process and on identifying potential problems in questionnaires (e.g., ambiguities, comprehension and memory problems, or problems with response categories or with the questionnaire instructions). To remain open to the possibility of new themes, this “theme coding” process was complemented by building codes from the ground up, based on the analysis of the data (O’Brien et al, 2013; Willis, 2015). Both the initial theme coding system and the inductive approach are rooted within the tradition of content analysis (see Willis, 2015, for an overview of these approaches).…”

“…To improve inter-rater reliability and to ensure objectivity, two independent researchers coded part of the interviews in parallel and recorded their results in writing (Lundgrén-Laine & Salanterä, 2010). All themes or problems identified in this manner were then analyzed and discussed in the four-member research team (one female and three male researchers with a range of expertise and knowledge), which helped to ensure researcher reflexivity, decrease bias that may result from a single person’s interpretation and thereby increases the credibility of research findings (Hall, Long, Bermbach, Jordan, & Patterson, 2005; O’Brien et al, 2013). To further ensure the quality of interpretations, the data were searched specifically for cases and examples that may contradict the interpretations.…”

“…By helping us understand why people respond the way they do, through explanatory analysis that takes into account the broader context influencing the response process, cognitive interviewing is a method of choice for examining this aspect of validity (Ramirez, Teresi, Ogedegbe, & Williams, 2016; Zumbo & Hubley, 2017). In particular, this method makes it possible to examine problematic aspects of face validity (e.g., whether the instrument as a whole is coherent), content validity (including the suitability of response options and which items should be modified), and ease of usage from the respondent’s perspective (e.g., whether the questions are clear) (O’Brien, Bereket, Swinton, & Solomon, 2013).…”

“…To minimize problems with questionnaires and ensure data collection is matched to the capabilities of the target audience, cognitive interviewing is reported to be particularly valuable when working with frail and elderly persons, when applied to sensitive questions, and when investigating populations with cognitive and physical impairments whose dependency relationships favor socially desirable response behaviors (Kelle et al, 2014; Murtagh et al, 2007; O’Brien et al, 2013). Although all the above conditions do apply to the research presented here (Gubrium, 1997), no comprehensive investigations exist regarding cognitive processing among nursing home residents while they are responding to subjective quality of life assessment tools that also allow the consideration of personal preferences.…”