Towards generalizable entity-centric clinical coreference resolution

Abstract

Objective This work investigates the problem of clinical coreference resolution in a model that explicitly tracks entities, and aims to measure the performance of that model in both traditional in-domain train/test splits and cross-domain experiments that measure the generalizability of learned models. Methods The two methods we compare are a baseline mention-pair coreference system that operates over pairs of mentions with best-first conflict resolution and a mention-synchronous system that incrementally builds coreference chains. We develop new features that incorporate distributional semantics, discourse features, and entity attributes. We use two new coreference datasets with similar annotation guidelines – the THYME colon cancer dataset and the DeepPhe breast cancer dataset. Results The mention-synchronous system performs similarly on in-domain data but performs much better on new data. Part of speech tag features prove superior in feature generalizability experiments over other word representations. Our methods show generalization improvement but there is still a performance gap when testing in new domains. Discussion Generalizability of clinical NLP systems is important and under-studied, so future work should attempt to perform cross-domain and cross-institution evaluations and explicitly develop features and training regimens that favor generalizability. A performance-optimized version of the mention-synchronous system will be included in the open source Apache cTAKES software.

Natural Language Processing of Clinical Notes on Chronic Diseases: Systematic Review

“…On the other hand, out of 16 papers involving publicly available corpora, 12 exploit the Informatics for Integrating Biology and the Bedside (i2b2) datasets. The other 4 public datasets used are MIMIC-II [107], PhenoCHF [116], Temporal Histories of Your Medical Event (THYME), and Cancer Deep Phenotype Extraction (DeepPhe) [76].…”

“…Carrell et al [66] proposed an NLP system to process clinical text to identify breast cancer recurrences, while Castro et al [22] addressed the automated Breast Imaging-Reporting and Data System (BI-RADS) categories extraction from breast radiology reports. Miller et al [76] proposed a tool for coreference resolution in clinical texts evaluated within the domain (colon cancer) and between domains (breast cancer). Mykowiecka et al [77] propose a rule-based IE system evaluated on mammography reports.…”

Longitudinal Visual Analytics for Unpacking the Cancer Journey

“…With histories involving dozens of relevant notes (one dataset used in DeepPhe averaged 30 notes/patient, after clearly irrelevant notes were removed), manual expert review will not be sufficient for the large-scale analyses needed to drive innovation. Although advances in cross-document coreference 15 and other techniques currently being explored by the DeepPhe project show great promise in increasing the utility of clinical text, NLP is only a first step, providing an intermediate representation not directly consumable by end-users.…”

Clinical Natural Language Processing for Radiation Oncology: A Review and Practical Primer

“…In addition, as with other AI algorithms, developing supervised NLP machine learning models almost always requires large gold annotated data sets from which the algorithm can learn. There are few publicly available corpora of gold annotated clinical text 11,63,93,94,[118][119][120][121][122][123][124][125][126][127][128] and no publicly available corpora of radiation oncology clinical documentation. Although some labeled data in these public corpora are translatable to radiation oncology, such as disease site, signs and symptoms, and comorbidities, the highly unique terminology and jargon used frequently in radiation oncology but rarely in other medical domains, such as "gross/ clinical/planning tumor volume," "boost," "point A/B," and "Gy," may require large radiation oncologyespecific gold annotated corpora.…”