Stacked Generalization for Medical Concept Extraction from Clinical Notes

Kim, Young‐Jun; Riloff, Ellen

doi:10.18653/v1/w15-3807

Cited by 11 publications

(4 citation statements)

References 29 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To take full advantages of above individual machine learning-based methods, we use an ensemble learning method [48], a support vector machine (SVM) classifier, to merge all PHI instances predicted by them. We use LibSVM [49] as an implementation of SVM.…”

Section: Methodsmentioning

confidence: 99%

De-identification of clinical notes via recurrent neural network and conditional random field

Liu

Tang

Wang

et al. 2017

Journal of Biomedical Informatics

121

114

View full text Add to dashboard Cite

De-identification, identifying information from data, such as protected health information (PHI) present in clinical data, is a critical step to enable data to be shared or published. The 2016 Centers of Excellence in Genomic Science (CEGS) Neuropsychiatric Genome-scale and RDOC Individualized Domains (N-GRID) clinical natural language processing (NLP) challenge contains a de-identification track in de-identifying electronic medical records (EMRs) (i.e., track 1). The challenge organizers provide 1000 annotated mental health records for this track, 600 out of which are used as a training set and 400 as a test set. We develop a hybrid system for the de-identification task on the training set. Firstly, four individual subsystems, that is, a subsystem based on bidirectional LSTM (long-short term memory, a variant of recurrent neural network), a subsystem-based on bidirectional LSTM with features, a subsystem based on conditional random field (CRF) and a rule-based subsystem, are used to identify PHI instances. Then, an ensemble learning-based classifiers is deployed to combine all PHI instances predicted by above three machine learning-based subsystems. Finally, the results of the ensemble learning-based classifier and the rule-based subsystem are merged together. Experiments conducted on the official test set show that our system achieves the highest micro F1-scores of 93.07%, 91.43% and 95.23% under the “token”, “strict” and “binary token” criteria respectively, ranking first in the 2016 CEGS N-GRID NLP challenge. In addition, on the dataset of 2014 i2b2 NLP challenge, our system achieves the highest micro F1-scores of 96.98%, 95.11% and 98.28% under the “token”, “strict” and “binary token” criteria respectively, outperforming other state-of-the-art systems. All these experiments prove the effectiveness of our proposed method.

show abstract

Section: Methodsmentioning

confidence: 99%

De-identification of clinical notes via recurrent neural network and conditional random field

Liu

Tang

Wang

et al. 2017

Journal of Biomedical Informatics

121

114

View full text Add to dashboard Cite

show abstract

“…To avoid these shortcomings, we chose a fine-grained learning-based ensemble method: stacking. Following Kim et al [21], we combined the predictions of the rule-based model and learning-based models via stacked generalization. Specifically, the predicted PHI from submodels are fed into a binary SVM-based classifier to make the decision about which PHI is more likely to be correct.…”

Section: Ensemble Methodsmentioning

confidence: 99%

Re-examination of Rule-Based Methods in Deidentification of Electronic Health Records: Algorithm Development and Validation

et al. 2020

View full text Add to dashboard Cite

Background Deidentification of clinical records is a critical step before their publication. This is usually treated as a type of sequence labeling task, and ensemble learning is one of the best performing solutions. Under the framework of multi-learner ensemble, the significance of a candidate rule-based learner remains an open issue. Objective The aim of this study is to investigate whether a rule-based learner is useful in a hybrid deidentification system and offer suggestions on how to build and integrate a rule-based learner. Methods We chose a data-driven rule-learner named transformation-based error-driven learning (TBED) and integrated it into the best performing hybrid system in this task. Results On the popular Informatics for Integrating Biology and the Bedside (i2b2) deidentification data set, experiments showed that TBED can offer high performance with its generated rules, and integrating the rule-based model into an ensemble framework, which reached an F1 score of 96.76%, achieved the best performance reported in the community. Conclusions We proved the rule-based method offers an effective contribution to the current ensemble learning approach for the deidentification of clinical records. Such a rule system could be automatically learned by TBED, avoiding the high cost and low reliability of manual rule composition. In particular, we boosted the ensemble model with rules to create the best performance of the deidentification of clinical records.

show abstract

“…Clinical notes in particular, which may contain important contextual information regarding family history, subtleties of symptoms and treatments, early warning signs, lifestyle, and socioeconomic factors, have been increasingly used for predictive modeling of disease. [2][3][4][5][6][7][8][9][10][11][12][13][14][15] However, EHR presents several challenges as it contains data that is recorded at irregular intervals and with varying frequencies (Figure 1) 16 . This can happen for a variety of reasons-in an emergency department (ED) or intensive care setting (ICU), patients with more frequent visits might be sicker than their counterparts.…”

Section: Introductionmentioning

confidence: 99%

Filling the gaps: leveraging large language models for temporal harmonization of clinical text across multiple medical visits for clinical prediction

Choi,

Long,

Getzen

2024

Preprint

View full text Add to dashboard Cite

Electronic health records offer great promise for early disease detection, treatment evaluation, information discovery, and other important facets of precision health. Clinical notes, in particular, may contain nuanced information about a patient's condition, treatment plans, and history that structured data may not capture. As a result, and with advancements in natural language processing, clinical notes have been increasingly used in supervised prediction models. To predict long-term outcomes such as chronic disease and mortality, it is often advantageous to leverage data occurring at multiple time points in a patient's history. However, these data are often collected at irregular time intervals and varying frequencies, thus posing an analytical challenge. Here, we propose the use of large language models (LLMs) for robust temporal harmonization of clinical notes across multiple visits. We compare multiple state-of-the-art LLMs in their ability to generate useful information during time gaps, and evaluate performance in supervised deep learning models for clinical prediction.

show abstract

Stacked Generalization for Medical Concept Extraction from Clinical Notes

Cited by 11 publications

References 29 publications

De-identification of clinical notes via recurrent neural network and conditional random field

De-identification of clinical notes via recurrent neural network and conditional random field

Re-examination of Rule-Based Methods in Deidentification of Electronic Health Records: Algorithm Development and Validation

Filling the gaps: leveraging large language models for temporal harmonization of clinical text across multiple medical visits for clinical prediction

Contact Info

Product

Resources

About