AHRQ Series on Improving Translation of Evidence: Perceived Value of Translational Products by the AHRQ EPC Learning Health Systems Panel

Borsky, Amanda; Savitz, Lucy A.; Bindman, Andrew B.; Mossburg, Sarah E.; Thompson, Lee

doi:10.1016/j.jcjq.2019.08.002

Cited by 8 publications

(11 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An important question is what model to use to provide prognostic information to clinicians. Using your own data to inform future care is consistent with a learning health system [ 67 ]. The ideal situation is that clinical decision support (CDS) could supply the best prediction for a patient based on the most recent trends at the time.…”

Section: Discussionmentioning

confidence: 99%

Predicting clinical outcomes among hospitalized COVID-19 patients using both local and published models

Galanter

Rodríguez-Fernández

Chow

et al. 2021

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

Background Many models are published which predict outcomes in hospitalized COVID-19 patients. The generalizability of many is unknown. We evaluated the performance of selected models from the literature and our own models to predict outcomes in patients at our institution. Methods We searched the literature for models predicting outcomes in inpatients with COVID-19. We produced models of mortality or criticality (mortality or ICU admission) in a development cohort. We tested external models which provided sufficient information and our models using a test cohort of our most recent patients. The performance of models was compared using the area under the receiver operator curve (AUC). Results Our literature review yielded 41 papers. Of those, 8 were found to have sufficient documentation and concordance with features available in our cohort to implement in our test cohort. All models were from Chinese patients. One model predicted criticality and seven mortality. Tested against the test cohort, internal models had an AUC of 0.84 (0.74–0.94) for mortality and 0.83 (0.76–0.90) for criticality. The best external model had an AUC of 0.89 (0.82–0.96) using three variables, another an AUC of 0.84 (0.78–0.91) using ten variables. AUC’s ranged from 0.68 to 0.89. On average, models tested were unable to produce predictions in 27% of patients due to missing lab data. Conclusion Despite differences in pandemic timeline, race, and socio-cultural healthcare context some models derived in China performed well. For healthcare organizations considering implementation of an external model, concordance between the features used in the model and features available in their own patients may be important. Analysis of both local and external models should be done to help decide on what prediction method is used to provide clinical decision support to clinicians treating COVID-19 patients as well as what lab tests should be included in order sets.

show abstract

Section: Discussionmentioning

confidence: 99%

Predicting clinical outcomes among hospitalized COVID-19 patients using both local and published models

Galanter

Rodríguez-Fernández

Chow

et al. 2021

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

show abstract

“…A recent AHRQ stakeholder engagement project with leaders of LHS around the United States identified some challenges with applying the findings of SRs produced by the AHRQ EPC Program to healthcare decision making. 8 LHS leaders noted that findings of SRs, especially those including few NRSIs or without consideration of local health system data, may often not be as useful for LHS decision making. A chief concern raised was the perception that evidence summarized does not adequately apply to the LHS's typical patient population.…”

Section: Rationale For Updated Guidancementioning

confidence: 99%

“…A chief concern raised was the perception that evidence summarized does not adequately apply to the LHS's typical patient population. 8,9 Including and considering the findings of NRSIs appropriately in EPC evidence syntheses has the potential to increase the applicability of EPC Program SRs to healthcare decision making by LHS and by healthcare decision makers more broadly.…”

Section: Rationale For Updated Guidancementioning

confidence: 99%

Inclusion of Nonrandomized Studies of Interventions in Systematic Reviews of Intervention Effectiveness: An Update

Saldanha¹,

Skelly²,

Ley³

et al. 2022

View full text Add to dashboard Cite

Introduction: Nonrandomized studies of interventions (NRSIs) are observational or experimental studies of the effectiveness and/or harms of interventions, in which participants are not randomized to intervention groups. There is increasingly widespread recognition that advancements in the design and analysis of NRSIs allow NRSI evidence to have a much more prominent role in decision making, and not just as ancillary evidence to randomized controlled trials (RCTs). Objective: To guide decisions about inclusion of NRSIs for addressing the effects of interventions in systematic reviews (SRs), this chapter updates the 2010 guidance on inclusion of NRSIs in Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) SRs. The chapter focuses on considerations for decisions to include or exclude NRSIs in SRs. Methods: In November 2020, AHRQ convened a 20-member workgroup that comprised 13 members representing 8 of 9 AHRQ-appointed EPCs, 3 AHRQ representatives, 1 independent consultant with expertise in SRs, and 3 representatives of the AHRQ-appointed Scientific Resource Center. The workgroup received input from the full EPC Program regarding the process and specific issues through discussions at a virtual meeting and two online surveys regarding challenges with NRSI inclusion in SRs. One survey focused on current practices by EPCs regarding NRSI inclusion in ongoing and recently completed SRs. The other survey focused on the appropriateness, completeness, and usefulness of existing EPC Program methods guidance. The workgroup considered the virtual meeting and survey input when identifying aspects of the guidance that needed updating. The workgroup used an informal method for generating consensus about guidance. Disagreements were resolved through discussion. Results: We outline considerations for the inclusion of NRSIs in SRs of intervention effectiveness. We describe the strengths and limitations of RCTs, study design features and types of NRSIs, and key considerations for making decisions about inclusion of NRSIs (during the stages of topic scoping and refinement, SR team formation, protocol development, SR conduct, and SR reporting). We discuss how NRSIs may be applicable for the decisional dilemma being addressed in the SR, threats to the internal validity of NRSIs, as well as various data sources and advanced analytic methods that may be used in NRSIs. Finally, we outline an approach to incorporating NRSIs within an SR and key considerations for reporting. Conclusion: The main change from the previous guidance is the overall approach to decisions about inclusion of NRSIs in EPC SRs. Instead of recommending NRSI inclusion only if RCTs are insufficient to address the Key Question, this updated guidance handles NRSI evidence as a valuable source of information and lays out important considerations for decisions about the inclusion of NRSIs in SRs of intervention effectiveness. Different topics may require different decisions regarding NRSI inclusion. This guidance is intended to improve the utility of the final product to end-users. Inclusion of NRSIs will increase the scope, time, and resources needed to complete SRs, and NRSIs pose potential threats to validity, such as selection bias, confounding, and misclassification of interventions. Careful consideration must be given to both concerns.

show abstract

“…And yet, AHRQ EPC reports can be hard to access and understand. To better fulfill its side of the producer‐consumer equation, AHRQ has recently increased efforts to determine how to adapt systematic review methods to meet the needs and timelines of a broader audience of evidence consumers 1,2 . Simultaneously, the agency is also attempting to ensure at least a subset of its public data systems more closely conform to the FAIR Guiding Principles, which call for scientific data to be Findable, Accessible, Interoperable, and Reusable 3 …”

Section: Introductionmentioning

confidence: 99%

“…To better fulfill its side of the producer-consumer equation, AHRQ has recently increased efforts to determine how to adapt systematic review methods to meet the needs and timelines of a broader audience of evidence consumers. 1,2 Simultaneously, the agency is also attempting to ensure at least a subset of its public data systems more closely conform to the FAIR Guiding Principles, which call for scientific data to be Findable, Accessible, Interoperable, and Reusable. 3 This Brief Methods Note critiques the current method and describes the development of a next generation (NxGen) AHRQ EPC Effective Health Care website that allows end-users of all types to find the evidence they need through data visualizations and other interactive displays.…”

Section: Introductionmentioning

confidence: 99%

NxGen evidence: Redesigning the Agency for Healthcare Research and Quality Effective Health Care website to promote engagement, interactivity, and usability of systematic reviews

Reid

Guíse

Fiordalisi

et al. 2020

Research Synthesis Methods

View full text Add to dashboard Cite

Evidence-based decision-making is predicated on the ability of users to find and comprehend results from systematic review. Evidence producers have an obligation to support evidence users in this process. The Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) program-a producer of rigorous and comprehensive systematic reviews for two decades-has set a gold standard for reliability in health evidence reviews in the United States. It has recently begun a program of active support for evidence dissemination and uptake beyond mere publication of lengthy reports. This Brief Methods Note critiques the current paper-based format for systematic reviews and describes the development of a next generation (NxGen) AHRQ EPC Effective Health Care website. This redesigned platform will allow end-users of all types to find and share the evidence they need through data visualizations and other interactive displays. Several design principles guided the development of NxGen to make systematic review findings more accessible, customizable, adaptable, interactive, and shareable. NxGen will include visualizations for report results that are expressed as meta-analyses as well as those with narrative syntheses, through forest or bubble plots, respectively. Visual and interactive evidence heat maps are also planned. The NxGen version of the Effective Health Care website is planned to go live in the latter half of 2020 or early 2021.

show abstract

AHRQ Series on Improving Translation of Evidence: Perceived Value of Translational Products by the AHRQ EPC Learning Health Systems Panel

Cited by 8 publications

References 16 publications

Predicting clinical outcomes among hospitalized COVID-19 patients using both local and published models

Predicting clinical outcomes among hospitalized COVID-19 patients using both local and published models

Inclusion of Nonrandomized Studies of Interventions in Systematic Reviews of Intervention Effectiveness: An Update

NxGen evidence: Redesigning the Agency for Healthcare Research and Quality Effective Health Care website to promote engagement, interactivity, and usability of systematic reviews

Contact Info

Product

Resources

About