Using ethnographic approaches to document, evaluate, and facilitate virtual community-engaged implementation research

Abstract

Background Community Advisory Boards (CABs) have been frequently used to engage diverse partners to inform research projects. Yet, evaluating the quality of engagement has not been routine. We describe a multi-method ethnographic approach documenting and assessing partner engagement in two “virtual” CABs, for which we conducted all meetings remotely. Methods Two research projects for increasing equitable COVID-19 testing, vaccination, and clinical trial participation for underserved communities involved remote CAB meetings. Thirty-three partners representing 17 community groups participated in 15 sessions across the two CABs facilitated by a social change organization. We developed ethnographic documentation forms to assess multiple aspects of CAB member engagement (e.g., time spent speaking, modality used, types of interactions). Documenters were trained to observe CAB sub-groups via virtual sessions. Debriefing with the documentation team after CAB meetings supported quality assurance and process refinement. CAB members completed a brief validated survey after each meeting to assess the quality and frequency of engagement. Content and rapid thematic analysis were used to analyze documentation data. Quantitative data were summarized as frequencies and means. Qualitative and quantitative findings were triangulated. Results A total of 4,540 interactions were identified across 15 meetings. The most frequent interaction was providing information (44%), followed by responding (37–38%). The quality and frequency of stakeholder engagement were rated favorably (average 4.7 of 5). Most CAB members (96%) reported good/excellent engagement. Specific comments included appreciation for the diversity of perspectives represented by the CAB members and suggestions for improved live interpretation. Debriefing sessions led to several methodological refinements for the documentation process and forms. Conclusion We highlight key strategies for documenting and assessing community engagement. Our methods allowed for rich ethnographic data collection that refined our work with community partners. We recommend ongoing trainings, including debriefing sessions and routinely reviewed assessment of data to strengthen meaningful community engagement.

Adaptation of the brainwriting premortem technique to inform the co-creation of COVID-19 testing strategies in underserved communities in South San Diego

“…The use of the brainwriting premortem exercise was feasible to complete in the context of our study, but it was not used as a stand-alone strategy to engage with our partners. We used brainwriting premortem in combination with monthly community and scientific advisory board meetings to develop a theory of change of COVID-19 disparities in our setting [ 11 ] and an iterative ethnographic assessment of the engagement process in the advisory board meetings [ 16 ]. Jolles-Perez and colleagues recently operationalized five principles of a co-created collaborative process to enhance implementation efforts that encompass equity, reflexivity, reciprocity and mutuality, transformative and personalized, and facilitating relationships [ 38 ].…”

“…The CO-CREATE program was designed to be responsive to the needs of the community and offers no-cost COVID-19 testing to patients and community members at the clinic site of the partnering (FQHC). Throughout our work we engaged with a multidisciplinary Community and Scientific Advisory Board to guide our program development and implementation [ 11 , 16 ].…”

Community-engaged optimization of COVID-19 rapid evaluation and testing experiences: roll-out implementation optimization trial

“…Aim 1 data collection will focus on evaluating the quality, extent, and content of partner engagement through the participatory CSAB activities. An established multimethod ethnographic approach [ 19 ] using a refined documentation form will be completed by trained research interns at each CSAB meeting, and a partner engagement survey based on validated measure from Goodman and colleagues [ 20 ] will be completed by all CSAB attendees after each session. Data from the partner engagement survey will be analyzed after each CSAB meeting to inform post-meeting debriefings with the Global ARC and research team.…”

“…Summarized data from these sources will be triangulate using a joint display analysis [ 21 , 22 ]. This methodology has been described in more detail in Rabin et al [ 19 ]. In addition, the research team will document adaptations made to the implementation strategies throughout the Aim 1 refinement process (and throughout Aim 2) using methods developed by Rabin et al and that was used in the phase 1 study [ 19 , 23 ].…”