Classification of Electronic Health Record–Related Patient Safety Incidents: Development and Validation Study

Abstract

Background It is assumed that the implementation of health information technology introduces new vulnerabilities within a complex sociotechnical health care system, but no international consensus exists on a standardized format for enhancing the collection, analysis, and interpretation of technology-induced errors. Objective This study aims to develop a classification for patient safety incident reporting associated with the use of mature electronic health records (EHRs). It also aims to validate the classification by using a data set of incidents during a 6-month period immediately after the implementation of a new EHR system. Methods The starting point of the classification development was the Finnish Technology-Induced Error Risk Assessment Scale tool, based on research on commonly recognized error types. A multiprofessional research team used iterative tests on consensus building to develop a classification system. The final classification, with preliminary descriptions of classes, was validated by applying it to analyze EHR-related error incidents (n=428) during the implementation phase of a new EHR system and also to evaluate this classification’s characteristics and applicability for reporting incidents. Interrater agreement was applied. Results The number of EHR-related patient safety incidents during the implementation period (n=501) was five-fold when compared with the preimplementation period (n=82). The literature identified new error types that were added to the emerging classification. Error types were adapted iteratively after several test rounds to develop a classification for reporting patient safety incidents in the clinical use of a high-maturity EHR system. Of the 427 classified patient safety incidents, interface problems accounted for 96 (22.5%) incident reports, usability problems for 73 (17.1%), documentation problems for 60 (14.1%), and clinical workflow problems for 33 (7.7%). Altogether, 20.8% (89/427) of reports were related to medication section problems, and downtime problems were rare (n=8). During the classification work, 14.8% (74/501) of reports of the original sample were rejected because of insufficient information, even though the reports were deemed to be related to EHRs. The interrater agreement during the blinded review was 97.7%. Conclusions This study presents a new classification for EHR-related patient safety incidents applicable to mature EHRs. The number of EHR-related patient safety incidents during the implementation period may reflect patient safety challenges during the implementation of a new type of high-maturity EHR system. The results indicate that the types of errors previously identified in the literature change with the EHR development cycle.

Preliminary Exploration of Main Elements for Systematic Classification Development: Case Study of Patient Safety Incidents

“…In a recent study [ 12 ], a classification for patient safety incident reporting associated with the use of a mature EHR was developed, which was validated using a data set of 501 patient safety incidents. Here, a mature EHR is defined according to the Electronic Medical Record Adoption Model, which was developed by Healthcare Information and Management Systems Society Analytics.…”

“…This universally recognized maturity model is an 8-stage model that reflects hospitals’ electronic medical record capabilities, ranging from a completely paper-based environment (stage 0) to a highly advanced paperless and digital patient record environment (stage 7). Regarding these data, the maturity level of the EHR system is 6 to 7 [ 12 ]. The classification development was based on research into commonly recognized error types.…”

“…To increase the reliability of qualitative research, applying a varied methodology is suggested [ 20 , 30 - 32 ]. For our use case [ 12 ], this involved retrieving clinical patient safety incident reporting data and conceptualizing technology-induced errors based on previous research, selecting relevant reports, analyzing incident data to define categories and hierarchies for the emerging classification, and applying a multidisciplinary panel within our research team to reach agreement over ongoing classification development ( Multimedia Appendix 1 ). We have already documented the study design for data analysis and validation [ 12 ], and in this paper, our study design concentrates on specifically capturing classification development characteristics ( Textbox 1 ) and related observations from the use case of our classification development.…”

“…Moreover, a recent study summarized that organizations today do not have rigorous, real-time approaches for routinely assessing the safety of EHRs or for identifying safety hazards [ 11 ]. This constitutes the underlying rationale for why we have recently developed and validated a classification of technology-induced error types from a social-technical perspective [ 12 ]. A guiding assumption in our research was that despite previous research introducing relevant themes in this area, further research is needed to capture the fast-developing clinical working environment.…”

“…To benefit future classification work, in this paper, we present preliminary results on methodological considerations based on previous research evidence and our case research on classifying EHR-related patient safety incidents [ 12 ]. Our preliminary results are intended to inform future research on classification development research protocols with different use cases.…”