The effectiveness and acceptability of evidence synthesis summary formats for clinical guideline development groups: A mixed-methods systematic review

Abstract

Introduction : Clinical guideline development often involves a rigorous synthesis of evidence involving multidisciplinary stakeholders with different priorities and knowledge of evidence synthesis; this makes communicating findings complex. Summary formats are typically used to communicate the results of evidence syntheses, however, there is little consensus on which formats are most effective and acceptable for different stakeholders. Methods This mixed-methods systematic review (MMSR) aimed to evaluate the effectiveness and acceptability (e.g., usability preferences, and attitudes and preferences towards) of evidence synthesis summary formats for GDG members. We followed the PRISMA 2020 guideline and Joanna Briggs Institute Manual for Evidence Synthesis for MMSRs. We searched six databases (inception to April 20, 2021) for randomised controlled trials (RCTs), RCTs with a qualitative component, and qualitative studies. Screening, data extraction, and quality appraisal was performed in duplicate. Qualitative findings were synthesised using meta-aggregation and quantitative findings are described narratively. Results We identified 17,240 citations and screened 54 full-text articles, resulting in 22 eligible articles (20 unique studies): 4 articles reported the results of 5 RCTs, one of which also had a qualitative component. The other 18 articles discussed the results of 16 qualitative studies. Therefore we had 5 trials and 17 qualitative studies to extract data from. Studies were geographically heterogeneous and included a variety of stakeholders and summary formats. All 5 RCTs assessed knowledge or understanding with 3 reporting improvement with newer formats. The qualitative analysis identified 6 themes: ‘presenting information’, ‘tailoring content for end users’, ‘trust in synthesis producers’, ‘knowledge requirements’, ‘quality of included studies’, and ‘properly contextualising findings’. Across these themes, the synthesis resulted in 130 recommendations for practice. Nine recommendations were supported by both quantitative and qualitative evidence and 121 by only qualitative. A majority focused on how to present information (n = 68) and tailor content for different end users (n = 24). Conclusions This MMSR provides guidance on how to improve evidence summary structure and layout. This can be used by synthesis producers to better communicate to GDGs. Study findings will inform the co-creation of evidence summary format prototypes based on GDG member’s needs. Word Count: 345 Registration: The protocol for this project was previously published and the project was pre-registered on Open Science Framework. [1, 2]

Co-design workshops to develop evidence synthesis summary formats for use by clinical guideline development groups

“…Although there was overall agreement with the MMSR recommendations (9), one area of debate was around the use of recommendations or 'author's conclusions.' Cochrane's guidance for plain language summaries does not advocate for providing recommendations.…”

“…Some research (26-28), including our MMSR (9), posited that a one-size-ts-all format may not be feasible due to individual differences in knowledge bases and priorities from the multidisciplinary members in clinical guideline development groups. In fact, we critiqued in our MMSR that the included qualitative studies often did not indicate a person's role (e.g., clinician, patient, decision-maker) when reporting participant quotes.…”

“…The protocol for this study was previously preregistered on the Open Science Framework (OSF). (10) Participants did not consent to sharing transcripts publicly and NVivo is a proprietary software therefore data sharing is not applicable to this article. Additional information is available on OSF.…”

“…Our protocol established a recruitment goal of at least 15 participants across three workshops. (10) Due to the variety of end-users and the number of MMSR recommendations (9), we aimed for six focus groups with at least four to seven participants each. We purposefully arranged groups to be multidisciplinary to avoid 'group-think,' aiming for a maximum of seven per group.…”

Co-design workshops to develop evidence synthesis summary formats for use by clinical guideline development groups

“…We recently conducted a mixed methods systematic review (MMSR) [8,9] to identify the most effective and acceptable summary format(s) for different end-users involved in clinical GDGs. Our review findings covered a range of types of evidence syntheses, end-users, and summary formats.…”