Making Health Research Matter: A Call to Increase Attention to External Validity

Huebschmann, Amy G.; Leavitt, Ian M.; Glasgow, Russell E.

doi:10.1146/annurev-publhealth-040218-043945

Cited by 58 publications

(33 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These concerns reflected a very broad spectrum of issues rather than a single barrier described in depth. Descriptive themes relating to implementation were less well developed than the themes relating to methodology, unsurprisingly given the focus on internal validity and study design that exist in evidence-based healthcare and the scant attention paid to issues of context and external validity [33]. The importance of institutional support and context were novel findings not apparent in traditionally conducted systematic literature reviews of HHIs such as our Cochrane review [2] which focused on internal threats to validity.…”

Section: Discussionmentioning

confidence: 94%

Applying thematic synthesis to interpretation and commentary in epidemiological studies: identifying what contributes to successful interventions to promote hand hygiene in patient care

et al. 2020

View full text Add to dashboard Cite

ObjectivesHand hygiene is considered the most important preventive measure for healthcare-associated infections, but adherence is suboptimal. We previously undertook a Cochrane Review that demonstrated that interventions to improve adherence are moderately effective. Impact varied between organisations and sites with the same intervention and implementation approaches. This study seeks to explore these differences.MethodsA thematic synthesis was applied to the original authors’ interpretation and commentary that offered explanations of how hand hygiene interventions exerted their effects and suggested reasons why success varied. The synthesis used a published Cochrane Review followed by three-stage synthesis.ResultsTwenty-one papers were reviewed: 11 randomised, 1 non-randomised and 9 interrupted time series studies. Thirteen descriptive themes were identified. They reflected a range of factors perceived to influence effectiveness. Descriptive themes were synthesised into three analytical themes: methodological explanations for failure or success (eg, Hawthorne effect) and two related themes that address issues with implementing hand hygiene interventions: successful implementation needs leadership and cooperation throughout the organisation (eg, visible managerial support) and understanding the context and aligning the intervention with it drives implementation (eg, embedding the intervention into wider patient safety initiatives).ConclusionsThe analytical themes help to explain the original authors’ perceptions of the degree to which interventions were effective and suggested new directions for research: exploring ways to avoid the Hawthorne effect; exploring the impact of components of multimodal interventions; the use of theoretical frameworks for behaviour change; potential to embed interventions into wider patient safety initiatives; adaptations to demonstrate sustainability; and the development of systematic approaches to implementation. Our findings corroborate studies exploring the success or failure of other clinical interventions: context and leadership are important.

show abstract

Section: Discussionmentioning

confidence: 94%

Applying thematic synthesis to interpretation and commentary in epidemiological studies: identifying what contributes to successful interventions to promote hand hygiene in patient care

et al. 2020

View full text Add to dashboard Cite

show abstract

“…It is recognised in health and social sciences that elegant studies can gain undue prestige despite their failings (Ioannidis 2005;Smaldino and McElreath 2016;Camerer et al 2018;Huebschmann et al 2019). Our own numerous examples (e.g., Sheil 1995Sheil , 1996Sheil et al 1999Sheil et al , 2016Sheil et al , 2019Sheil and Wunder 2002;Makarieva et al 2014), and many others, suggest similar processes in other sciences including ecology, environment and climate.…”

Section: Resultsmentioning

confidence: 99%

Interpreting forest diversity-productivity relationships: volume values, disturbance histories and alternative inferences

Sheil

Bongers

2020

For. Ecosyst.

View full text Add to dashboard Cite

Understanding the relationship between stand-level tree diversity and productivity has the potential to inform the science and management of forests. History shows that plant diversity-productivity relationships are challenging to interpret-and this remains true for the study of forests using non-experimental field data. Here we highlight pitfalls regarding the analyses and interpretation of such studies. We examine three themes: 1) the nature and measurement of ecological productivity and related values; 2) the role of stand history and disturbance in explaining forest characteristics; and 3) the interpretation of any relationship. We show that volume production and true productivity are distinct, and neither is a demonstrated proxy for economic values. Many stand characteristics, including diversity, volume growth and productivity, vary intrinsically with succession and stand history. We should be characterising these relationships rather than ignoring or eliminating them. Failure to do so may lead to misleading conclusions. To illustrate, we examine the study which prompted our concerns -Liang et al. (Science 354:aaf8957, 2016)-which developed a sophisticated global analysis to infer a worldwide positive effect of biodiversity (tree species richness) on "forest productivity" (stand level wood volume production). Existing data should be able to address many of our concerns. Critical evaluations will improve understanding.

show abstract

“…In order to prove the calibration of the nomogram, clinical data was collected from different institutions. As is well known, the internal validity associated with the explanation of the results, and the external validity related to the generalizability of the results [17,18]. Through the internal and external validation data set analysis, the calibration of our nomogram has been proved to be highly consistent, which was more accurate than APS-III (B) and SOFA scores.…”

Section: Discussionmentioning

confidence: 59%

Development and Validation of a Nomogram to Predict the Mortality Risk in Elderly Patients With ARF：A Retrospective Study in the Mimic-III Database

Chen¹,

Weng²,

Hou³

et al. 2020

Preprint

View full text Add to dashboard Cite

Background: To explore the risk factors of prognosis in elderly patients with acute respiratory failure (ARF), and to develop a nomogram model to predict the short-term mortality risk of ARF.Methods: A total of 1432 patients were included in this study from MIMIC-III database. 759 patients were categorized into the training set and 673 patients were categorized into the validation set. Demographical, laboratory variables, SOFA score and APS-III score were collected within the first 24 h after the ICU admission. The univariate and multivariate logistic regression were used to identify risk factors from the training data set. A nomogram model was developed to predict the mortality risk of ARF patients within 30 days according to the risk factors.Results: Multivariate logistic regression analysis showed that the heart rate, respiratory rate, systolic pressure, SPO2, albumin and 24 h urine output were independent prognostic factors for 30-day mortality in ARF patients. A nomogram was established based on above independent prognostic factors. This nomogram had C-index of 0.741 (95% CI: 0.7058–0.7766), and the C-index was 0.687 (95%CI: 0.6458-0.7272) in the validation set. The calibration curves both in training and validation set were close to the ideal model. The SOFA had a C-index of 0.653 and the APS-III had a C-index of 0.707 in predicting 30-day mortality. The predictive performance of our nomogram is better than the SOFA score and APS-III score. Conclusions: Our nomogram performed better than APS-III and SOFA scores and should be useful as decision support on the prediction of mortality risk in elderly patients with ARF.

show abstract

Making Health Research Matter: A Call to Increase Attention to External Validity

Cited by 58 publications

References 71 publications

Applying thematic synthesis to interpretation and commentary in epidemiological studies: identifying what contributes to successful interventions to promote hand hygiene in patient care

Applying thematic synthesis to interpretation and commentary in epidemiological studies: identifying what contributes to successful interventions to promote hand hygiene in patient care

Interpreting forest diversity-productivity relationships: volume values, disturbance histories and alternative inferences

Development and Validation of a Nomogram to Predict the Mortality Risk in Elderly Patients With ARF：A Retrospective Study in the Mimic-III Database

Contact Info

Product

Resources

About