Improving Radiology Report Quality by Rapidly Notifying Radiologist of Report Errors

Minn, Matthew; Zandieh, Arash R.; Filice, Ross W.

doi:10.1007/s10278-015-9781-9

Cited by 22 publications

(12 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has been well demonstrated in similar cases such as flagging report errors or providing feedback on exam duration that analogous continuous quality assurance feedback results in consistent error correction and lower baseline error rates. Immediate and consistent feedback applications hold the potential to improve awareness and baseline functioning and are essential in a field where error must be minimized [16, 17].…”

Section: Discussionmentioning

confidence: 99%

Effectiveness of Deep Learning Algorithms to Determine Laterality in Radiographs

Filice

Frantz

2019

J Digit Imaging

Self Cite

View full text Add to dashboard Cite

Develop a highly accurate deep learning model to reliably classify radiographs by laterality. Digital Imaging and Communications in Medicine (DICOM) data for nine body parts was extracted retrospectively. Laterality was determined directly if encoded properly or inferred using other elements. Curation confirmed categorization and identified inaccurate labels due to human error. Augmentation enriched training data to semi-equilibrate classes. Classification and object detection models were developed on a dedicated workstation and tested on novel images. Receiver operating characteristic (ROC) curves, sensitivity, specificity, and accuracy were calculated. Study-level accuracy was determined and both were compared to human performance. An ensemble model was tested for the rigorous use-case of automatically classifying exams retrospectively. The final classification model identified novel images with an ROC area under the curve (AUC) of 0.999, improving on previous work and comparable to human performance. A similar ROC curve was observed for per-study analysis with AUC of 0.999. The object detection model classified images with accuracy of 99% or greater at both image and study level. Confidence scores allow adjustment of sensitivity and specificity as needed; the ensemble model designed for the highly specific use-case of automatically classifying exams was comparable and arguably better than human performance demonstrating 99% accuracy with 1% of exams unchanged and no incorrect classification. Deep learning models can classify radiographs by laterality with high accuracy and may be applied in a variety of settings that could improve patient safety and radiologist satisfaction. Rigorous use-cases requiring high specificity are achievable.

show abstract

Section: Discussionmentioning

confidence: 99%

Effectiveness of Deep Learning Algorithms to Determine Laterality in Radiographs

Filice

Frantz

2019

J Digit Imaging

Self Cite

View full text Add to dashboard Cite

show abstract

“…Several custom-developed software solutions aiming to decrease this error type have been devised. These include color-coded laterality discriminator crosschecks prior to report signing [16] and algorithms comparing report content to patients' Health Level 7 metadata [17]. However, no software solution is available to detect discrepancies between findings and impression report sections in real time.…”

Section: Discussionmentioning

confidence: 99%

Revealing the most common reporting errors through data mining of the report proofreading process

et al. 2020

View full text Add to dashboard Cite

Objectives To investigate the most common errors in residents’ preliminary reports, if structured reporting impacts error types and frequencies, and to identify possible implications for resident education and patient safety. Material and methods Changes in report content were tracked by a report comparison tool on a word level and extracted for 78,625 radiology reports dictated from September 2017 to December 2018 in our department. Following data aggregation according to word stems and stratification by subspecialty (e.g., neuroradiology) and imaging modality, frequencies of additions/deletions were analyzed for findings and impression report section separately and compared between subgroups. Results Overall modifications per report averaged 4.1 words, with demonstrably higher amounts of changes for cross-sectional imaging (CT: 6.4; MRI: 6.7) than non-cross-sectional imaging (radiographs: 0.2; ultrasound: 2.8). The four most frequently changed words (right, left, one, and none) remained almost similar among all subgroups (range: 0.072–0.117 per report; once every 9–14 reports). Albeit representing only 0.02% of analyzed words, they accounted for up to 9.7% of all observed changes. Subspecialties solely using structured reporting had substantially lower change ratios in the findings report section (mean: 0.2 per report) compared with prose-style reporting subspecialties (mean: 2.0). Relative frequencies of the most changed words remained unchanged. Conclusion Residents’ most common reporting errors in all subspecialties and modalities are laterality discriminator confusions (left/right) and unnoticed descriptor misregistration by speech recognition (one/none). Structured reporting reduces overall error rates, but does not affect occurrence of the most common errors. Increased error awareness and measures improving report correctness and ensuring patient safety are required. Key Points • The two most common reporting errors in residents’ preliminary reports are laterality discriminator confusions (left/right) and unnoticed descriptor misregistration by speech recognition (one/none). • Structured reporting reduces the overall the error frequency in the findings report section by a factor of 10 (structured reporting: mean 0.2 per report; prose-style reporting: 2.0) but does not affect the occurrence of the two major errors. • Staff radiologist review behavior noticeably differs between radiology subspecialties.

show abstract

“…Per our institutional report quality standards, the reporting errors were divided into 2 categories: item errors and content errors 8 . Item errors refer to the lack of the outpatient or hospitalization number or the clinical diagnosis of a patient.…”

Section: Methodsmentioning

confidence: 99%

“…Item errors refer to the lack of the outpatient or hospitalization number or the clinical diagnosis of a patient. Content errors include absence of measurement data, sex‐related errors, and a wrong orientation 8 . Typical sex‐related errors include mention of male organs in the US reports of female patients and vice versa 8 .…”

Section: Methodsmentioning

confidence: 99%

An Intelligent Reminder System Reduces Deficiencies and Errors in Ultrasound Reports

Zhang

Jin

et al. 2020

J of Ultrasound Medicine

View full text Add to dashboard Cite

Objectives We designed a computer‐based, integrated intelligent reminder system to reduce the deficiencies and errors in ultrasound (US) reports. In this study, we assessed the performance of this system and evaluated its impact on the quality of US reporting. Methods Ultrasound reporting deficiencies or errors were divided into 2 categories: missing items (including outpatient or inpatient number and clinical diagnosis) and content errors (including measurement data, sex‐related, and laterality errors). The intelligent reminder system was designed in Visual Basic for Applications (Microsoft Corporation, Redmond, WA) and integrated with the US system. It automatically detects reporting errors before printing of the report and provides real‐time prompts for correction of the errors. We compared the US reporting deficiencies and errors during the 20 months before and after implementation of the system. Results Before implementation of the system, deficiencies/errors were detected in 2.26% (8841 of 391,230) of US reports compared with 0.12% (530 of 444,215) of reports after implementation of the system (P < .0001). After adoption of the system, the reported item deficiencies were improved more than the content deficiencies, with the most notable improvement in clinical diagnosis. Sex‐related errors were reduced from 7 cases to nil after use of the intelligent reminder system. No laterality errors were found before and after the implementation of the system. Conclusions The intelligent reminder system within the US system significantly reduced deficiencies and errors, improving the quality of the report.

show abstract

Improving Radiology Report Quality by Rapidly Notifying Radiologist of Report Errors

Cited by 22 publications

References 18 publications

Effectiveness of Deep Learning Algorithms to Determine Laterality in Radiographs

Effectiveness of Deep Learning Algorithms to Determine Laterality in Radiographs

Revealing the most common reporting errors through data mining of the report proofreading process

An Intelligent Reminder System Reduces Deficiencies and Errors in Ultrasound Reports

Contact Info

Product

Resources

About