CHAracteristics of research studies that iNfluence practice: a GEneral survey of Canadian orthopaedic Surgeons (CHANGES): a pilot survey

Abstract

BackgroundEvidence Based Medicine (EBM) is increasingly being applied to inform clinical decision-making in orthopaedic surgery. Despite the promotion of EBM in Orthopaedic Surgery, the adoption of results from high quality clinical research seems highly unpredictable and does not appear to be driven strictly by randomized trial data. The objective of this study was to pilot a survey to determine if we could identify surgeon opinions on the characteristics of research studies that are perceived as being most likely to influence clinical decision-making among orthopaedic surgeons in Canada.MethodsA 28-question electronic survey was distributed to active members of the Canadian Orthopaedic Association (COA) over a period of 11 weeks. The questionnaire sought to analyze the influence of both extrinsic and intrinsic characteristics of research studies and their potential to influence practice patterns. Extrinsic factors included the perceived journal quality and investigator profiles, economic impact, peer/patient/industry influence and individual surgeon residency/fellowship training experiences. Intrinsic factors included study design, sample size, and outcomes reported. Descriptive statistics are provided.ResultsOf the 109 members of the COA who opened the survey, 95 (87%) completed the survey in its entirety. The overall response rate was 11% (95/841). Surgeons achieved consensus on the influence of three key designs on their practices: 1) randomized controlled trials 94 (99%), 2) meta-analysis 83 (87%), and 3) systematic reviews 81 (85%). Sixty-seven percent of surgeons agreed that studies with sample sizes of 101–500 or more were more likely to influence clinical practice than smaller studies (n = <100). Factors other than design influencing adoption included 1) reputation of the investigators (99%) and 2) perceived quality of the journal (75%).ConclusionAlthough study design and sample size (i.e. minimum of 100 patients) have some influence on clinical decision making, surgeon respondents are equally influenced by investigator reputation and perceived journal quality. At present, continued emphasis on the generation of large, methodologically sound clinical trials remains paramount to translating research findings to clinical practice changes. Specific to this pilot survey, strategies to solicit more widespread responses will be pursued.

An international survey to identify the intrinsic and extrinsic factors of research studies most likely to change orthopaedic practice

“…While efforts were employed to determine the importance of individual study factors on informing a surgeon’s practice, not all extrinsic and intrinsic factors surveyed are of equal importance. 9 The bulk of respondents were from North America and Europe. As such, it is possible that factors identified to be important for informing change of practice may not be applicable outside of these jurisdictions.…”

“… 9 Feedback from this pilot trial was incorporated to decrease the timeline for accepting survey respondents and adjust the rollout period of the survey to avoid coinciding with major conferences. 9 The proposed survey was reviewed with the communications department of the Journal of Bone and Joint Surgery (JBJS) (Am) and further revisions were made to adjust question stem wording to a more global sample population with an unbiased tone. Additionally, country of practice was added to the demographic section of the questionnaire.…”

“… 7 , 8 High-quality evidence (randomised-controlled trials (RCTs), systematic reviews and meta-analyses of RCTs) is recognised as a requirement for orthopaedic surgeons to consider a change of practice. 9 , 10 Mattila et al 11 have shown that high-quality RCTs can in fact initiate evidence-based practice changes in orthopaedic surgery. Yet, only 3% of the orthopaedic literature represents RCTs.…”

“… 13 , 14 We have previously shown in our pilot survey of the Canadian Orthopaedic Association (COA) membership that three key study designs (RCTs, meta-analysis and systematic reviews) are most likely to influence a change in practice in Canada. 9 Furthermore, studies with sample sizes greater than 100 participants and studies published by high-profile investigators in journals of perceived high quality are significant factors when Canadian orthopaedic surgeons interpret evidence to guide practice. 9 …”

“… 9 Furthermore, studies with sample sizes greater than 100 participants and studies published by high-profile investigators in journals of perceived high quality are significant factors when Canadian orthopaedic surgeons interpret evidence to guide practice. 9 …”