Development and Implementation of A LIS-Based Validation System for Autoverification Toward Zero Defects in The Automated Reporting of Laboratory Test Results

Abstract

BackgroundValidation of the autoverification function is the most critical step to confirm its effectiveness before use. It is crucial to verify whether the programmed algorithm follows the expected logic and produces the expected results. In recent years, this process has always been centered on the assessment of human-machine consistency and mostly takes the form of manual recording, which is a time-consuming activity with inherent subjectivity and arbitrariness, and cannot guarantee a comprehensive, timely and continuous effectiveness evaluation of the autoverification function. To overcome these inherent limitations, we independently developed and implemented a laboratory information system (LIS)-based validation system for autoverification.MethodsWe developed a correctness verification and integrity validation method (hereinafter referred to as the "new method") in the form of a human-machine dialogue. The system records the personnel’s review steps and determines if the human-machine review results are consistent. If they are inconsistent, the laboratory personnel analyze the reasons for the inconsistency according to the system prompts, add to or modify the rules, reverify, and finally improve the accuracy of autoverification.ResultsThe validation system was successfully established and implemented. For a dataset consisting of 833 rules for 30 assays, 782 rules (93.87%) were successfully verified in the correctness verification phase, and 51 rules were deleted due to execution errors. In the integrity validation phase, 24 projects were easily verified, while the other 6 projects still required the addition of new rules or changes to the rule settings. From setting the rules to the automated reportion, the time difference between manual validation and the new method, was statistically significant (χ2=11.06, p=0.0009), with the new method greatly reducing validation time. Since 2017, the new method has been used in 32 laboratories, and 15.8 million reports have been automatically reviewed and issued without a single clinical complaint.ConclusionTo the best of our knowledge, this is the first report to realize autoverification validation in the form of a human-machine interaction.The new method can effectively control the risks of autoverification, shorten time consumption, and improve the efficiency of laboratory verification.