Quality, methods, and recommendations of systematic reviews on measures of evidence-based practice: an umbrella review

Roberge‐Dao, Jacqueline; Maggio, Lauren A.; Zaccagnini, Marco; Rochette, Annie; Shikako‐Thomas, Keiko; Boruff, Jill; Thomas, Aliki

doi:10.11124/jbies-21-00118

Cited by 5 publications

(13 citation statements)

References 114 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The included SRs were published between 2006 and 2020 and the population scope varied from all healthcare professionals ( n = 3) to specific healthcare professional groups (OTs, n = 2; nurses, n = 2; physical therapists, n = 1; family medicine residents, n = 1; medical students, n = 1). Detailed information on the included SRs is found elsewhere 42 …”

Section: Resultsmentioning

confidence: 99%

“…We conducted a secondary analysis of qualitative data obtained in the context of a previously published umbrella review 42 that aimed to compare SRs on EBP measures. This article aims to answer the second objective of the protocol registered at PROSPERO CRD42020160874.…”

Section: Methodsmentioning

confidence: 99%

“…The umbrella review was conducted following the Joanna Briggs Institute methodology for umbrella reviews and according to the Preferred Reporting Items for SRs and meta‐analyses guidelines 45,46 . Complete information on the methodology of the previously published review is available elsewhere 42 …”

Section: Methodsmentioning

confidence: 99%

“…Though authors of included SRs call for more rigorous psychometric evaluations, they fail to explain this statement or provide reference to methodological guidance. These findings warrant questions such as ‘what is a more rigorous psychometric evaluation?’ and ‘what is considered strong evidence?’ The criteria used to determine the strength of psychometric evidence and the adequacy of measures varied for each SR and reflects authors' understanding of validity 42 . The Standards of Psychological and Educational Testing which are considered the ‘gold standard’ for assessment practices in the social sciences (e.g., education, psychology and employment) were only referenced by two SRs 41,54 (although only one 54 adequately applied the methods).…”

Section: Psychometrics In Ebp Measurementmentioning

confidence: 99%

“…31 Our team conducted an umbrella review on systematic reviews (SRs) of psychometric properties of EBP measures. 42 Specifically, we estimated the quality of SRs on EBP measures across healthcare professions and identified differences between reviews regarding the recommended measures and the approaches used to assess the adequacy of EBP measures. The quality of the 10 included SRs was low; none of the SRs assessed the quality of primary studies or adhered to SR methodological guidelines.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Challenges and future directions in the measurement of evidence‐based practice: Qualitative analysis of umbrella review findings

Roberge‐Dao

Maggio

Zaccagnini

et al. 2022

Evaluation Clinical Practice

Self Cite

View full text Add to dashboard Cite

Rationale, Aims and Objectives An important aspect of scholarly discussions about evidence‐based practice (EBP) is how EBP is measured. Given the conceptual and empirical developments in the study of EBP over the last 3 decades, there is a need to better understand how to best measure EBP in educational and clinical contexts. The aim of this study was to identify and describe the main challenges, recommendations for practice, and areas of future research in the measurement of EBP across the health professions as reported by systematic reviews (SRs). Methods We conducted a secondary analysis of qualitative data obtained in the context of a previously published umbrella review that aimed to compare SRs on EBP measures. Two reviewers independently extracted excerpts from the results and discussion/conclusion sections of the 10 included SRs that aligned with the three research aims. An iterative six‐phase reflexive thematic analysis according to Braun and Clarke was conducted. Results Our thematic analysis produced five themes describing the main challenges associated with measuring EBP, four themes outlining main recommendations for practice, and four themes representing areas of future research. Challenges include limited psychometric testing and validity evidence for existing EBP measures; limitations with the self‐report format; lack of construct clarity of EBP measures; inability to capture the complexity of the EBP process and outcomes; and the context‐specific nature of EBP measures. Reported recommendations for practice include acknowledging the multidimensionality of EBP; adapting EBP measures to the context and re‐examining the validity argument; and considering the feasibility and acceptability of measures. Areas of future research included the development of comprehensive, multidimensional EBP measures and the need for expert consensus on the operationalization of EBP. Conclusions This study suggests that existing measures may be insufficient in capturing the multidimensional, contextual and dynamic nature of EBP. There is a need for a clear operationalization of EBP and an improved understanding and application of validity theory.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Psychometrics In Ebp Measurementmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Challenges and future directions in the measurement of evidence‐based practice: Qualitative analysis of umbrella review findings

Roberge‐Dao

Maggio

Zaccagnini

et al. 2022

Evaluation Clinical Practice

Self Cite

View full text Add to dashboard Cite

show abstract

Identifying candidate items for a prototype index on propensity to integrate research evidence into clinical decision-making in rehabilitation

Roberge-Dao,

Thomas,

Rochette

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

PurposeExisting measures of evidence-based practice (EBP) in rehabilitation provide a fragmented interpretation of EBP competencies as performance is rated on discrete domains without a harmonized measure to represent the multidimensionality of EBP. Building on previous work, this study aimed to provide evidence that a brief multidimensional index can be formed to determine a clinician’s propensity to integrate research evidence into decision-making to inform the subsequent development process.MethodsUsing a Canadian dataset of occupational and physical therapists (n=127) who responded to a survey containing 70 frequently used items to measure EBP (representing six domains), one item per key EBP domain was selected using Rasch measurement theory and expert consensus. A preliminary scoring algorithm was developed for testing purposes. The interpretability of the prototype index was examined across characteristics of the sample and compared to full EBP measures using generalized estimating equations.ResultsFive items were selected for inclusion in the prototype index representing the dimensions ofuse of research evidence, self-efficacy, resources, attitudes,andactivities. Testing demonstrated that the prototype behaves consistently regardless of age groups, gender, and setting, and provides comparable information to full EBP measures.ConclusionThis study provides promising preliminary evidence to justify continuing the index development process. The benefits of having a global index of EBP as opposed to having multiple domain-specific measures are discussed in this paper.

show abstract

Improving Clarity and Interpretability of Items in a Bilingual Index of Propensity to Integrate Research Evidence into Clinical Decision-Making in Rehabilitation

Roberge-Dao,

Mayo,

Rochette

et al. 2024

Preprint

View full text Add to dashboard Cite

PurposeTo contribute evidence for the clarity and interpretability of a new five-item bilingual multidimensional index of a rehabilitation clinician’s propensity to integrate research evidence into clinical decision-making.MethodsThis study was conducted in three sequential steps: (1) We conducted a focus group with occupational therapists, physical therapists, and researchers to review the items and response options for clarity, consistency, and interval properties and agree on equivalency in English and French. (2) We conducted cognitive interviews whereby clinicians elaborated on their interpretation of the item, comprehensibility of items, and appropriateness of response options. Accepted modifications were integrated and tested with subsequent participants. (3) We conducted an online survey to validate the English and French equivalency of response options on a 0 to 100 scale.ResultsDuring the qualitative revision process (one focus group with 7 participants followed by 27 interviews), the index was revised 12 times with substantial modifications to theuse of research evidenceandattitudesitems.ConclusionThis research increases the clinical relevance and reduces measurement error of this brief index which can inform on individual or organizational factors influencing a clinician’s propensity of integrating research evidence into decision-making and ultimately, improve rehabilitation outcomes.

show abstract

Quality, methods, and recommendations of systematic reviews on measures of evidence-based practice: an umbrella review

Cited by 5 publications

References 114 publications

Challenges and future directions in the measurement of evidence‐based practice: Qualitative analysis of umbrella review findings

Challenges and future directions in the measurement of evidence‐based practice: Qualitative analysis of umbrella review findings

Identifying candidate items for a prototype index on propensity to integrate research evidence into clinical decision-making in rehabilitation

Improving Clarity and Interpretability of Items in a Bilingual Index of Propensity to Integrate Research Evidence into Clinical Decision-Making in Rehabilitation

Contact Info

Product

Resources

About