Automated classification of primary care patient safety incident report content and severity using supervised machine learning (ML) approaches

Evans, H. P.; Αναστασίου, Αθανάσιος; Edwards, Adrian; Hibbert, Peter; Makeham, Meredith; Luz, Saturnino; Sheikh, Aziz; Donaldson, Liam; Carson‐Stevens, Andrew

doi:10.1177/1460458219833102

Cited by 35 publications

(30 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several studies focused on classifying the type and severity of patient safety incident reports using data collected by different sources such as universities [ 75 ], and incident reporting systems such as Advanced Incident Management Systems (across Australia) and Riskman [ 67 , 75 , 76 ]. Others analyzed hospital clinical notes internally (manually annotated by clinicians and a quality committee) and data retrieved from patient safety organizations to identify adverse incidents such as delayed medication [ 68 ], fall risks [ 47 , 67 ], near misses, patient misidentification, spelling errors, and ambiguity in clinical notes [ 109 ].…”

Section: Resultsmentioning

confidence: 99%

Role of Artificial Intelligence in Patient Safety Outcomes: Systematic Literature Review

2020

View full text Add to dashboard Cite

Background Artificial intelligence (AI) provides opportunities to identify the health risks of patients and thus influence patient safety outcomes. Objective The purpose of this systematic literature review was to identify and analyze quantitative studies utilizing or integrating AI to address and report clinical-level patient safety outcomes. Methods We restricted our search to the PubMed, PubMed Central, and Web of Science databases to retrieve research articles published in English between January 2009 and August 2019. We focused on quantitative studies that reported positive, negative, or intermediate changes in patient safety outcomes using AI apps, specifically those based on machine-learning algorithms and natural language processing. Quantitative studies reporting only AI performance but not its influence on patient safety outcomes were excluded from further review. Results We identified 53 eligible studies, which were summarized concerning their patient safety subcategories, the most frequently used AI, and reported performance metrics. Recognized safety subcategories were clinical alarms (n=9; mainly based on decision tree models), clinical reports (n=21; based on support vector machine models), and drug safety (n=23; mainly based on decision tree models). Analysis of these 53 studies also identified two essential findings: (1) the lack of a standardized benchmark and (2) heterogeneity in AI reporting. Conclusions This systematic review indicates that AI-enabled decision support systems, when implemented correctly, can aid in enhancing patient safety by improving error detection, patient stratification, and drug management. Future work is still needed for robust validation of these systems in prospective and real-world clinical environments to understand how well AI can predict safety outcomes in health care settings.

show abstract

Section: Resultsmentioning

confidence: 99%

Role of Artificial Intelligence in Patient Safety Outcomes: Systematic Literature Review

2020

View full text Add to dashboard Cite

show abstract

“…Alternatively, non-linear SVM uses the function to map the input vector to a high-dimension or infinite-dimension vector space and determines the hyperplane in the new space to classify the data points [ 30 ]. It has been previously observed that SVMs have consistently outperformed many other classifiers in text categorization problems, and they are less prone to the imbalanced data sets [ 31 ].…”

Section: Methodsmentioning

confidence: 99%

Automatic Incident Triage in Radiation Oncology Incident Learning System

et al. 2020

View full text Add to dashboard Cite

The Radiotherapy Incident Reporting and Analysis System (RIRAS) receives incident reports from Radiation Oncology facilities across the US Veterans Health Affairs (VHA) enterprise and Virginia Commonwealth University (VCU). In this work, we propose a computational pipeline for analysis of radiation oncology incident reports. Our pipeline uses machine learning (ML) and natural language processing (NLP) based methods to predict the severity of the incidents reported in the RIRAS platform using the textual description of the reported incidents. These incidents in RIRAS are reviewed by a radiation oncology subject matter expert (SME), who initially triages some incidents based on the salient elements in the incident report. To automate the triage process, we used the data from the VHA treatment centers and the VCU radiation oncology department. We used NLP combined with traditional ML algorithms, including support vector machine (SVM) with linear kernel, and compared it against the transfer learning approach with the universal language model fine-tuning (ULMFiT) algorithm. In RIRAS, severities are divided into four categories; A, B, C, and D, with A being the most severe to D being the least. In this work, we built models to predict High (A & B) vs. Low (C & D) severity instead of all the four categories. Models were evaluated with macro-averaged precision, recall, and F1-Score. The Traditional ML machine learning (SVM-linear) approach did well on the VHA dataset with 0.78 F1-Score but performed poorly on the VCU dataset with 0.5 F1-Score. The transfer learning approach did well on both datasets with 0.81 F1-Score on VHA dataset and 0.68 F1-Score on the VCU dataset. Overall, our methods show promise in automating the triage and severity determination process from radiotherapy incident reports.

show abstract

“…Each free-text report was coded using the multi-axial PISA classification system, which is aligned to the World Health Organisation International Classification for Patient Safety, and has been extensively used to characterise patient safety incidents data in primary care [9][10][11][12][13].…”

Section: Data Codingmentioning

confidence: 99%

Patient-safety incidents during COVID-19 health crisis in France: An exploratory sequential multi-method study in primary care

Fournier

Amélineau

Hild

et al. 2021

European Journal of General Practice

Self Cite

View full text Add to dashboard Cite

show abstract

Automated classification of primary care patient safety incident report content and severity using supervised machine learning (ML) approaches

Cited by 35 publications

References 34 publications

Role of Artificial Intelligence in Patient Safety Outcomes: Systematic Literature Review

Role of Artificial Intelligence in Patient Safety Outcomes: Systematic Literature Review

Automatic Incident Triage in Radiation Oncology Incident Learning System

Patient-safety incidents during COVID-19 health crisis in France: An exploratory sequential multi-method study in primary care

Contact Info

Product

Resources

About