Machine learning-driven clinical decision support system for concept-based searching: a field trial in a Norwegian hospital

Abstract

Background Natural language processing (NLP) based clinical decision support systems (CDSSs) have demonstrated the ability to extract vital information from patient electronic health records (EHRs) to facilitate important decision support tasks. While obtaining accurate, medical domain interpretable results is crucial, it is demanding because real-world EHRs contain many inconsistencies and inaccuracies. Further, testing of such machine learning-based systems in clinical practice has received limited attention and are yet to be accepted by clinicians for regular use. Methods We present our results from the evaluation of an NLP-driven CDSS developed and implemented in a Norwegian Hospital. The system incorporates unsupervised and supervised machine learning combined with rule-based algorithms for clinical concept-based searching to identify and classify allergies of concern for anesthesia and intensive care. The system also implements a semi-supervised machine learning approach to automatically annotate medical concepts in the narrative. Results Evaluation of system adoption was performed by a mixed methods approach applying The Unified Theory of Acceptance and Use of Technology (UTAUT) as a theoretical lens. Most of the respondents demonstrated a high degree of system acceptance and expressed a positive attitude towards the system in general and intention to use the system in the future. Increased detection of patient allergies, and thus improved quality of practice and patient safety during surgery or ICU stays, was perceived as the most important advantage of the system. Conclusions Our combined machine learning and rule-based approach benefits system performance, efficiency, and interpretability. The results demonstrate that the proposed CDSS increases detection of patient allergies, and that the system received high-level acceptance by the clinicians using it. Useful recommendations for further system improvements and implementation initiatives are reducing the quantity of alarms, expansion of the system to include more clinical concepts, closer EHR system integration, and more workstations available at point of care.

Exploring the role of professional identity in the implementation of clinical decision support systems—a narrative review

“…explainability and transparency of decision outcomes ( n = 43) [ 8 – 12 , 68 – 70 , 72 , 76 , 80 , 83 , 84 , 87 , 88 , 90 , 91 , 93 , 100 , 106 , 107 , 111 , 114 , 121 , 134 , 141 , 142 , 147 – 149 , 151 , 152 , 154 – 160 ] Ease of use ( n = 39) [ 9 , 10 , 17 , 24 , 30 , 34 , 68 , 69 , 71 , 73 , 74 , 76 – 78 , 86 , 87 , 90 , 91 , 93 , 96 , 104 , 106 , 107 , 110 , 111 , 115 , …”

“…Continuous feedback loops between health care professionals, IT staff, and implementation managers can help identify unforeseen threats to professional identity and necessary adjustments to the implementation plan. The review found that perceived identity threats particularly need to be addressed among highly specialized physicians to account for their knowledge-intensive skills, expertise, and clinical workflows [ 24 , 96 ]. In addition, the purpose of CDSS implementation and information about how it aligns with organizational strategic goals and individual professional development should be clearly and continuously communicated at all stages of implementation.…”

“…In this context, research that explicitly focuses on AI-based CDSSs becomes increasingly important as they become more relevant in medical practice. In fact, five of the studies included in our research, conducted over the last 3 years, examined factors related to the adoption and implementation of AI-based CDSS [ 73 , 74 , 96 , 205 , 206 ]. AI-based CDSSs extend to full automation and can discover new relationships and make predictions based on learned patterns [ 97 ].…”

Improving Neurology Clinical Care With Natural Language Processing Tools

“…16 NLP can aid in automating the extraction of patient demographics, medical history, symptoms, and diagnostic information from textual data, enabling improved clinical decision-making and personalized treatment approaches. 17 Furthermore, NLP tools can assist in the identification and monitoring of disease progression, detecting patterns in large-scale datasets, predicting treatment outcomes, and identifying adverse effects. 18 In addition, NLP can support the creation of neurology-specific ontologies and standardized vocabularies, facilitating better information exchange and interoperability across different health care systems and research institutions.…”

Methodology for the development and application of clinical decisions support information technologies with consideration of civil-legal grounds

“…There are 2 types of CDSIT [31]: knowledge-driven (based on scientific knowledge; consisting of an information base of rules of the "if-then" type, a mechanism of logical conclusion, and a mechanism of communication) and non-knowledge-driven (based on the results of processing the collected statistical data using mathematical methods, for example, machine learning [32]). Considering the presence of clear civil-legal grounds that regulate the possibility of making this or that medical decision, on the basis of which it is possible and expedient to build exactly the rules of the "if-then" type, the purpose of our research is the development and application of knowledge-driven, rule-based, clinical decisions support information technologies with consideration civil-legal grounds.…”