Guidelines for Conducting Virtual Cognitive Interviews During a Pandemic

Shepperd, James A.; Pogge, Gabrielle; Hunleth, Jean; Ruiz, Sienna; Waters, Erika A.

doi:10.2196/25173

Cited by 6 publications

(2 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Methodologic standards and best practices for cognitive interviewing [ 13 , 14 ] should, in future iterations, address considerations for conducting digitally supported cognitive interviews. Lastly, there is a need to test, refine and scale recently described technological innovations that support inclusion of study participants who do not have access to computer hardware or internet connectivity [ 15 ].…”

Section: Discussionmentioning

confidence: 99%

Considerations for conducting and reporting digitally supported cognitive interviews with children and adults

Fry

Mitchell

Wiener

2021

J Patient Rep Outcomes

View full text Add to dashboard Cite

Background Cognitive interviewing is a well-established qualitative method used to develop and refine PRO measures. A range of digital technologies including phone, web conferencing, and electronic survey platforms can be leveraged to support the conduct of cognitive interviewing in both children and adults. These technologies offer a potential solution to enrolling underrepresented populations, including those with rare conditions, functional limitations and geographic or socioeconomic barriers. In the aftermath of the COVID-19 pandemic, the use of digital technologies for qualitative interviewing will remain essential. However, there is limited guidance about adapting cognitive interviewing procedures to allow for remote data capture, especially with children. Methods Synthesizing the literature and our research experiences during the COVID-19 pandemic, we examine considerations for implementing digitally supported cognitive interviews with children, adolescents, and adults. We offer recommendations to optimize data quality and empirical rigor and illustrate the application of these recommendations in an ongoing cognitive interviewing study to develop and refine a new pediatric PRO measure. Results Good research practices must address participant and researcher preparation for study-related procedures and should anticipate and pre-emptively manage technological barriers. Field notes should detail interview context, audio/video cues, and any impact of technological difficulties on data quality. The approaches we recommend have been tested in an ongoing cognitive interviewing study that is enrolling children/adolescents with cGVHD ages 5–17 and their caregivers [NCT 04044365]. The combined use of telephone and videoconferencing to conduct cognitive interviews remotely is feasible and acceptable and yields meaningful data to improve the content validity of our new PRO measure of cGVHD symptom bother. Conclusion Digitally supported cognitive interviewing procedures will be increasingly employed. Remote data collection can accelerate accrual, particularly in multi-site studies, and may allow for interviewer personnel and data management to be centralized within a coordinating center, thus conserving resources. Research is needed to further test and refine techniques for remote cognitive interviewing, particularly in traditionally underrepresented populations, including children and non-English speakers. Expansion of international standards to address digitally supported remote qualitative data capture appears warranted.

show abstract

Section: Discussionmentioning

confidence: 99%

Considerations for conducting and reporting digitally supported cognitive interviews with children and adults

Fry

Mitchell

Wiener

2021

J Patient Rep Outcomes

View full text Add to dashboard Cite

show abstract

“…We argue that open-ended probes in online pretesting can help to detect potential response quality problems when using opt-in panels to pretest survey questions in English and Spanish. Notably, online cognitive testing has become even more common in recent times due to safety restrictions on face-to-face interactions during the COVID-19 pandemic (Shepperd et al 2021;Upadhyay and Lipkovich 2020). Thus, scholarship needs to identify challenges associated with data quality issues using online opt-in panels.…”

Section: Introductionmentioning

confidence: 99%

Identifying Data Quality Challenges in Online Opt-In Panels Using Cognitive Interviews in English and Spanish

Trejo

Meyers

Martinez

et al. 2022

Journal of Official Statistics

View full text Add to dashboard Cite

In this article, we evaluate how the analysis of open-ended probes in an online cognitive interview can serve as a metric to identify cases that should be excluded due to disingenuous responses by ineligible respondents. We analyze data collected in 2019 via an online opt-in panel in English and Spanish to pretest a public opinion questionnaire (n = 265 in English and 199 in Spanish). We find that analyzing open-ended probes allowed us to flag cases completed by respondents who demonstrated problematic behaviors (e.g., answering many probes with repetitive textual patterns, by typing random characters, etc.), as well as to identify cases completed by ineligible respondents posing as eligible respondents (i.e., non-Spanish-speakers posing as Spanish-speakers). These findings indicate that data collected for multilingual pretesting research using online opt-in panels likely require additional evaluations of data quality. We find that open-ended probes can help determine which cases should be replaced when conducting pretesting using opt-in panels. We argue that open-ended probes in online cognitive interviews, while more time consuming and expensive to analyze than close-ended questions, serve as a valuable method of verifying response quality and respondent eligibility, particularly for researchers conducting multilingual surveys with online opt-in panels.

show abstract

Orthodontists’ use of remote monitoring platforms pre-, amid, and post-COVID-19: a survey study

Logan,

Riedy,

Hargett

et al. 2024

BMC Oral Health

View full text Add to dashboard Cite

Background Did the COVID-19 pandemic affect orthodontists’ use of remote monitoring platforms? The goal of this research was to examine orthodontists’ experiences implementing remote monitoring platforms before, during, and after the initial COVID-19 lockdown. Methods In this descriptive cross-sectional survey study, an electronic, anonymous questionnaire consisting of a series of 31 short-answer and multiple-choice questions was administered to an international sampling of practicing orthodontists. The target population in the study included currently practicing orthodontists who were graduates of an accredited orthodontic residency program. Participants were recruited in 2021 through collaboration with the American Association of Orthodontists (AAO) Partners in Research Program and the Harvard School of Dental Medicine Orthodontic Alumni Association. Descriptive analysis was conducted, reporting frequency (N and %) distributions for each question. The questionnaire aimed to describe whether orthodontists incorporated remote monitoring platforms into their practices, their experiences doing so, and if the COVID-19 pandemic influenced their use of these resources. Results Orthodontists’ use of remote monitoring platforms was negligible prior to the pandemic; however, a quarter of surveyed orthodontists began using a remote monitoring platform during COVID-19 and nearly all respondents plan to continue using remote monitoring for the foreseeable future. Approximately half of orthodontists believe most patients’ treatment progress can be monitored to the standard of care between in-person orthodontic appointments using remote monitoring platforms. Half of the orthodontists who do not currently use a remote monitoring platform in their practice are interested in learning more about how to implement one. Conclusions The COVID-19 pandemic led to an increase in the interest and adoption of remote monitoring platforms in orthodontic practices. Most orthodontists had not incorporated remote monitoring platforms into their practices prior to the COVID-19 pandemic. However, this study revealed that a subset of orthodontists utilized the pandemic as motivation to incorporate remote monitoring into their practices and an additional group of orthodontists were interested in incorporating one in the future. Remote monitoring platforms garnered interest and importance with the arrival of the COVID-19 pandemic and may only have an increasing role in the field in years to come.

show abstract

Guidelines for Conducting Virtual Cognitive Interviews During a Pandemic

Cited by 6 publications

References 4 publications

Considerations for conducting and reporting digitally supported cognitive interviews with children and adults

Considerations for conducting and reporting digitally supported cognitive interviews with children and adults

Identifying Data Quality Challenges in Online Opt-In Panels Using Cognitive Interviews in English and Spanish

Orthodontists’ use of remote monitoring platforms pre-, amid, and post-COVID-19: a survey study

Contact Info

Product

Resources

About