Development of an Artificial Intelligence–Based Automated Recommendation System for Clinical Laboratory Tests: Retrospective Analysis of the National Health Insurance Database

Abstract

Background Laboratory tests are considered an essential part of patient safety as patients’ screening, diagnosis, and follow-up are solely based on laboratory tests. Diagnosis of patients could be wrong, missed, or delayed if laboratory tests are performed erroneously. However, recognizing the value of correct laboratory test ordering remains underestimated by policymakers and clinicians. Nowadays, artificial intelligence methods such as machine learning and deep learning (DL) have been extensively used as powerful tools for pattern recognition in large data sets. Therefore, developing an automated laboratory test recommendation tool using available data from electronic health records (EHRs) could support current clinical practice. Objective The objective of this study was to develop an artificial intelligence–based automated model that can provide laboratory tests recommendation based on simple variables available in EHRs. Methods A retrospective analysis of the National Health Insurance database between January 1, 2013, and December 31, 2013, was performed. We reviewed the record of all patients who visited the cardiology department at least once and were prescribed laboratory tests. The data set was split into training and testing sets (80:20) to develop the DL model. In the internal validation, 25% of data were randomly selected from the training set to evaluate the performance of this model. Results We used the area under the receiver operating characteristic curve, precision, recall, and hamming loss as comparative measures. A total of 129,938 prescriptions were used in our model. The DL-based automated recommendation system for laboratory tests achieved a significantly higher area under the receiver operating characteristic curve (AUROCmacro and AUROCmicro of 0.76 and 0.87, respectively). Using a low cutoff, the model identified appropriate laboratory tests with 99% sensitivity. Conclusions The developed artificial intelligence model based on DL exhibited good discriminative capability for predicting laboratory tests using routinely collected EHR data. Utilization of DL approaches can facilitate optimal laboratory test selection for patients, which may in turn improve patient safety. However, future study is recommended to assess the cost-effectiveness for implementing this model in real-world clinical settings.

Laboratory Demand Management Strategies—An Overview

“…Currently, few published articles deal with the issue of applying AI algorithms to laboratory test selection. Islam et al [ 56 , 57 ] have published two such studies, one of which in this issue, where they developed a deep learning algorithm based on retrospective patient data to predict appropriate laboratory tests. Xu et al [ 58 ] aimed to identify superfluous tests in existing lab orders by estimating normal test results within a retrospective dataset.…”

Clinical application of intelligent technologies and integration in medical laboratories

“…Deep learning (DL) models can accurately and effectively identify proper laboratory tests. AI models developed based on DL had the good discriminative abilities required for further laboratory testing and for extrapolation using routinely collected electronic health record data [31,32]. Xu et al [33] 86 -iLABMED conducted a retrospective study of 116 637 inpatients using ML models and confirmed that redundant non-essential medical tests had been performed.…”

“…Deep learning (DL) models can accurately and effectively identify proper laboratory tests. AI models developed based on DL had the good discriminative abilities required for further laboratory testing and for extrapolation using routinely collected electronic health record data [31, 32]. Xu et al.…”

Realizing the promise of big data: how Taiwan can help the world reduce medical errors and advance precision medicine

“…Using deep learning-based systems and retrospective NHIRD data for a given condition, one team developed automated laboratory test recommendation systems drawing on EHR data. This approach could reduce both over-and underutilization of tests to improve patient care and prevent unnecessary costs [38,39]. Similarly, private corporations have joined the effort.…”