Efficiency and safety increases after the implementation of a multi‐institutional automated plan check tool at our institution

Abstract

Purpose: The plan check tool (PCT) is the result of a multi-institutional collaboration to jointly develop a flexible automated plan checking framework designed with the versatility to be shared across collaborating facilities while supporting the individual differences between practices. We analyze the effect that PCT has had on the efficiency and effectiveness of initial chart checks at our institution.Methods and Materials: Data on errors identified during initial chart checks were acquired during two time periods: before the introduction of PCT in the clinic (6/24/ 2015 to 7/31/2015, 187 checks) and post-clinical release (4/14/2016 to 5/2/2016, 186 checks). During each time period, human plan checkers were asked to record all issues that they either manually detected or that were detected by PCT as well as the amount of time, less breaks, or interruptions, it took to check each plan.Results: After the clinical release of PCT, there was a statistically significant decrease in the number of issues recorded by the human plan checkers both related to checks explicitly performed by PCT (13 vs 50, P < 0.001) and in issues identified overall (127 vs 200, P < 0.001). The mean and medium time for a plan check decreased by 20%. Conclusions:The use of a multi-institutional, configurable, automated plan checking tool has resulted in both substantial gains in efficiency and moving error detection to earlier points in the planning process, decreasing their likelihood that they reach the patient. The sizeable startup effort needed to create this tool from scratch was mitigated by the sharing, and subsequent co-development, of software code from a peer institution. K E Y W O R D Sautomation, plan checking, quality and safety, treatment planning

Graph‐based risk assessment and error detection in radiation therapy

“…We expect that clinical use of our chart review solution will provide data on the frequency of different types of errors encountered as has been reported in other studies 12,13,24,23 . This will permit subsequent data mining for a more complete risk assessment, and concomitant risk reduction 25 .…”

“…A key recommendation of AAPM Task Group 275, the objective of which was to provide recommendations on the physics chart review, is the development of automated tools to assist with the chart review task 1 . Most current automated and semi‐automated chart review tools are similar to an automated checklist to verify the integrity of variables in a specific module or their consistency across software components used to create the treatment plan based on predefined rules 5–14 . Several studies have shown the use of such tools to increase efficiency and patient safety and to reduce the rate of plan revisions, avoidable safety events, and patient delays 10,11,13 …”

Combining automatic plan integrity check (APIC) with standard plan document and checklist method to reduce errors in treatment planning

“…Even with the integrated planning system and RO‐EMR system, some communication errors listed in Table 4 still require manual checks such as the use of bolus and instruction of isocenter shifts. A recent publication from MSKCC, under the integrated environment of the treatment planning and RO‐EMT systems, indicated that their automatic plan check tool reported three outputs for their checked items: pass, flag, and report 15 . The items labeled with “flag” or “report” require manual check.…”

“…In order to detect errors made in the planning process, research has focused on designing manual and automated checks throughout this process 5–16 . Automated checks focus mainly on the technical integrity of a plan and on dosimetric metrics that are established for each treatment site.…”

Creating a treatment plan report should be mandated as a minimum standard practice for patient care and QA documentation

“…In reality, automated programs are mostly rule based and require continuous adaptation to ever changing clinical practices. Published studies, [3][4][5] including one from my institution, 5 showed that hybrid automated checks and manual checks catch…”