The ability of triggers to retrospectively predict potentially preventable adverse events in a sample of deceased patients

Klein, Dorthe O; Rennenberg, Roger J M W; Koopmans, Richard P.; Prins, Martin H.

doi:10.1016/j.pmedr.2017.10.016

Cited by 9 publications

(15 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This possibly results in considerable useless time-consuming scrutinizing of records by expensive specialists. Solutions to improve efficacy could be the use of more reproducible triggers (such as the objective ones), combining triggers with patient characteristics, or fully computerized trigger detection by “data mining” software [23]. Before implementing such adaptations, we suggest thorough research concerning the exact performance and costs for finding preventable AEs.…”

Section: Discussionmentioning

confidence: 99%

“…Therefore, the definition of trigger 1 was adapted to “patient has been admitted in the previous three months for a reason related to the index admission”. Example of cases with corresponding triggers is explained in further detail elsewhere [23]. Each record was reviewed twice by one of the nurses, once prior to the study in the context of their regular work as screeners and once during this study.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

The Harvard medical practice study trigger system performance in deceased patients

Klein

Rennenberg

Koopmans

et al. 2019

BMC Health Serv Res

Self Cite

View full text Add to dashboard Cite

BackgroundTo detect possible threats to quality and safety, multiple systems have been developed. One of them is retrospective chart review. A team of experts scrutinizes medical records, selected by trigger systems, to detect possible adverse events (AEs). The most important AEs and more hints for possible improvement of care appear in deceased patients. Using triggers in a sample of these patients might increase the performance and lower the burden of scrutinizing records without possible preventable AEs. The aim of this study was therefore to determine the performance of the trigger system in a sample of deceased patients and to calculate the specificity and the sensitivity of this trigger system for predicting AEs.MethodsWe performed a study in which the records of deceased patients were screened for triggers by a team of trained nurses. A sample of 100 medical records was randomly selected out of records which had been screened between 2012 and 2015 for the first time, prior to the study in 2016. For the determination of significant differences between the first and second screening, McNemar’s test of symmetry was used. Also, observed agreement, Cohen’s Kappa and prevalence-adjusted and-bias-adjusted-kappa (PABAK) statistics were calculated. This was done for the two trigger rounds on both any trigger present and for every trigger separately.ResultsThe observed agreement for any given trigger was 75% with a Kappa and PABAK of 0.5. For the individual triggers, the observed agreement was on average 90%. The corresponding Kappa was on average 0.42 (range: − 0.03-0.78) and the average PABAK was 0.8 (range: 0.44–0.92). Two adverse events were found in cases without triggers previously. The recalculated specificity and sensitivity for the original population were 58 and 92% respectively.ConclusionsFor the reproducibility of triggers it seems that some perform better than others, but on average this is to our opinion suboptimal. The low specificity implies that many records are selected without AEs. This leads to a high false-positive rate making this labour-intensive record review process costly. Therefore, research for better and more expedient systems is required.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

The Harvard medical practice study trigger system performance in deceased patients

Klein

Rennenberg

Koopmans

et al. 2019

BMC Health Serv Res

Self Cite

View full text Add to dashboard Cite

show abstract

“…Consequently, this would lead to a lower workload for the committee. 7 At the same time, there are a substantial number of cases, with potentially preventable AEs, not detected by machine learning (20% for CNN and FT algorithm, 18%…”

Section: Discussionmentioning

confidence: 99%

“…Another issue is the relatively high inter-rater variation and low reproducibility of trigger results and subsequently AE detection with current methods. [1][2][3][4][5][6][7] This suggests that these methods generate results that make translation to the clinical practice difficult.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Detecting adverse events in clinical care using natural language processing

Klein

Rennenberg

Heuvel³

et al. 2019

Preprint

View full text Add to dashboard Cite

Background Current methods for retrospective review of medical records require both time- and cost-wise a substantial effort. Therefore, we wanted to find the best method (based on natural language processing (NLP)) to select cases out of the medical records for further investigation in search for a (potentially preventable) adverse event (AE) to the decrease this effort. Methods The basic dataset consisted of 2987 medical records of patients who died during their hospitalization. To gain insight into the signal to noise ratio of the various resources, several subsets of our basic dataset were tested. Thereafter, we tested the scalability. After the best subset was chosen, several NLP algorithms were tested to select the best performing algorithm for the detecting of AEs. In the last experiment we tested the performance of the computer algorithms to predict potentially preventable AEs. The results of the NLP were compared with the outcome of the original retrospective medical record review. Results The dataset which contained he last three letters of the medical record showed the biggest potential. The scalability experiment showed that more data leads to a better performance of the algorithm. The best performing algorithm in the third test was the one based on support vector machine (SVM), with a precision of 79%, a negative predictive value (NPV) of 95% and a specificity of 85%. The results of the preventability experiment showed that the performance of the algorithms was almost equal to the results of the AEs. Conclusions In this study, we have shown that the SVM algorithm generates the most accurate results for the selection of cases for further investigation in the search for a (potentially preventable) AE. The sensitivity of the algorithms was around 75%. However, the SVM algorithm selected fewer cases to be examined for AEs compared to the original method. Consequently, this would lead to a lower workload for the committee. At the same time, there are a substantial number of cases, with potentially preventable AEs, not detected by machine learning.

show abstract

Rate of Preventable Mortality in Hospitalized Patients: a Systematic Review and Meta-analysis

et al. 2020

View full text Add to dashboard Cite

BACKGROUND:The number of preventable inpatient deaths in the USA is commonly estimated as between 44,000 and 98,000 deaths annually. Because many inpatient deaths are believed to be preventable, mortality rates are used for quality measures and reimbursement. We aimed to estimate the proportion of inpatient deaths that are preventable. METHODS: A systematic literature search of Medline, Embase, Web of Science, and the Cochrane Library through April 8, 2019, was conducted. We included case series of adult patients who died in the hospital and were reviewed by physicians to determine if the death was preventable. Two reviewers independently performed data extraction and study quality assessment. The proportion of preventable deaths from individual studies was pooled using a random-effects model. RESULTS: Sixteen studies met inclusion criteria. Eight studies of consecutive or randomly selected cohorts including 12,503 deaths were pooled. The pooled rate of preventable mortality was 3.1% (95% CI 2.2-4.1%). Two studies also reported rates of preventable mortality limited to patients expected to live longer than 3 months, ranging from 0.5 to 1.0%. In the USA, these estimates correspond to approximately 22,165 preventable deaths annually and 7150 deaths for patients with greater than 3month life expectancy. DISCUSSION: The number of deaths due to medical error is lower than previously reported and the majority occur in patients with less than 3-month life expectancy. The vast majority of hospital deaths are due to underlying disease. Our results have implications for the use of hospital mortality rates for quality reporting and reimbursement.

show abstract

The ability of triggers to retrospectively predict potentially preventable adverse events in a sample of deceased patients

Cited by 9 publications

References 25 publications

The Harvard medical practice study trigger system performance in deceased patients

The Harvard medical practice study trigger system performance in deceased patients

Detecting adverse events in clinical care using natural language processing

Rate of Preventable Mortality in Hospitalized Patients: a Systematic Review and Meta-analysis

Contact Info

Product

Resources

About