The UAB Informatics Institute and 2016 CEGS N-GRID de-identification shared task challenge

Abstract

Clinical narratives (the text notes found in patients’ medical records) are important information sources for secondary use in research. However, in order to protect patient privacy, they must be de-identified prior to use. Manual de-identification is considered to be the gold standard approach but is tedious, expensive, slow, and impractical for use with large-scale clinical data. Automated or semi-automated de-identification using computer algorithms is a potentially promising alternative. The Informatics Institute of the University of Alabama at Birmingham is applying de-identification to clinical data drawn from the UAB hospital’s electronic medical records system before releasing them for research. We participated in a shared task challenge by the Centers of Excellence in Genomic Science (CEGS) Neuropsychiatric Genome-Scale and RDoC Individualized Domains (N-GRID) at the de-identification regular track to gain experience developing our own automatic de-identification tool. We focused on the popular and successful methods from previous challenges: rule-based, dictionary-matching, and machine-learning approaches. We also explored new techniques such as disambiguation rules, term ambiguity measurement, and used multi-pass sieve framework at a micro level. For the challenge’s primary measure (strict entity), our submissions achieved competitive results (f-measures: 87.3%, 87.1%, and 86.7%). For our preferred measure (binary token HIPAA), our submissions achieved superior results (f-measures: 93.7%, 93.6%, and 93%). With those encouraging results, we gain the confidence to improve and use the tool for the real de-identification task at the UAB Informatics Institute.

A review of Automatic end-to-end De-Identification: Is High Accuracy the Only Metric?

“…De-identification system Machine learning S1 (Zhao, Zhang, Ma, and Li (2018)), S2 (Chen, Cullen, and Godwin (2015)) S3 (Dernoncourt, Lee, Uzuner, and Szolovits (2017)), S4 (Yadav, Ekbal, Saha, Pathak, and Bhattacharyya (2017)), S5 ), S6 ) Hybrid S7 (Yang and Garibaldi (2015)) S8 (Liu, Tang, Wang, and Chen (2017)) S9 (Lee, Dernoncourt, Uzuner, and Szolovits (2016)) S10 (Dehghan, Kovacevic, Karystianis, Keane, and Nenadic (2015)) S11 (Yang and Garibaldi (2015)) S12 (He, Guan, Cheng, Cen, and Hua (2015)) S13 (Liu, Chen, Tang, Wang, Chen, Li, Wang, Deng, and Zhu (2015)) S14 (Phuong and Chau (2016)) S15 (Bui, Wyatt, and Cimino (2017a)) S16 (Jiang, Zhao, He, Guan, and Jiang (2017)) S17 (Lee, Wu, Zhang, Xu, Xu, and Roberts (2017)) S18 (Shweta, Kumar, Ekbal, Saha, and Bhattacharyya (2016)) In this section, we outline the most significant achievement of automating end-toend de-identification system: improving accuracy. It has been argued that as far as de-identification is concerned, perfection cannot be achieved; however, 95% accuracy is considered to be the rule of thumb and universally accepted value ; ).…”

“…Unlike other medical data, such as that of the 2014 challenge, psychiatric data contains an abundance of information related to the patients such as places lived, jobs held, children's ages, hobbies, traumatic events, patients' relatives' relationship information, and pet names. This makes it a much more significant challenge to deidentify (Bui, Wyatt, and Cimino (2017b); ).…”

Re-examination of Rule-Based Methods in Deidentification of Electronic Health Records: Algorithm Development and Validation

“…In the 2016 i2b2 shared task, ensemble with rule-based models became more popular. Lee et al [12], Dehghan et al [13], Bui et al [14], and Liu et al [15] all employed rule-based models as a component of their hybrid systems. However, despite the wide use of rules, all the works did not investigate the effect of rule-based models in hybrid architecture.…”

A survey of automatic de-identification of longitudinal clinical narratives

“…De-identification of EHR can be done manually or automatically. Manual deidentification is time consuming, tedious, costly, can only be done by a restricted set of individuals allowed access to the original patient notes, and is subject to human error (Bui, Wyatt, and Cimino (2017); Dehghan, Kovacevic, Karystianis, Keane, and Nenadic (2015); Dernoncourt, Lee, Uzuner, and Szolovits (2017); He, Guan, Cheng, Cen, and Hua (2015)). There have been many attempts to develop an automatic system that will de-identify EHR with certainty.…”