Consensus Among Experts and Research Synthesis: A Comparison of Methods

Abstract

A comparison of two assessment methods, consensus among experts and research synthesis of the scientific literature, was performed using a surgical procedure, carotid endarterectomy (CE), as an example. These two methods have been widely advocated as being scientifically valid. While the comparison revealed a number of areas of general agreement, important differences between the two methods emerged. For example, 30-day mortality for asymptomatic patients was considered an effective outcome (ranked first) by the synthesis, but only "equivocal" (ranked third) of six major indicators reported by the consensus method. The synthesis results are also consistent with other literature reviews as well as with recent large-scale randomized trial results. A number of factors that could account for differences between the two methods were examined. Overall, use of consensus panels may be appropriate early in the development of an intervention where the evidence is sparse, while quantitative research synthesis is preferable when a number of high-quality studies have been performed.

Consensus conferences must include a systematic search and categorization of the evidence

“…However, we think it is important to emphasize that the recent advances in the methodology of systematic reviews should also play a role in the lbrmulation of clinical guidelines [24]. This can probably be done most easily by first performing a systematic review and then using it as a basis for guideline formulation [29].…”

Developing interdisciplinary maternity services policy in Canada. Evaluation of a consensus workshop

“…The opportunity that the PHCT funding provided was to test different models and ways of interprofessional collaboration. Holism/synthesis/responsiveness to local needs and key stakeholders [8] Need to understand the whole context to understand the contained meaning Self-organization is the process by which agentsmutually adapt behaviours to cope with changing internal and external environmental demands Thus they create the new structures and behaviours needed to meet changet [4,5].…”

“…A search for patterns implies attention to the flow of behaviour within systems rather than merely describing static behaviour [7]. Consensus was identified as the most appropriate method to synthesize research findings in this context [8].…”

A social choice approach to expert consensus panels

“…We are simply identifying the conditions that an expert panel ideally should meet. decision processes (e.g., McDonnell et al, 1996;Herrin et al, 1997;Shekelle et al, 1998;Wortman et al, 1998;Kahan et al, 1996;Bernstein et al, 1997;Shekelle et al, 1999). While these features of the process clearly have normative importance, the primary goal of these panels is not consistency, nor even high quality statements, but rather to reach an accurate judgment, an aspect that these empirical studies cannot address.…”