Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-Based Neural Networks

Blinov, Pavel; Avetisian, Manvel; Kokh, Vladimir; Umerenkov, Dmitry; Tuzhilin, Alexander

doi:10.1007/978-3-030-59137-3_11

Cited by 17 publications

(8 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With the development of natural language processing technology, there are many studies on the design of triage system using unstructured data [ 7 , 45 , 46 ]. The unstructured data used is mainly the chief complaint, because the chief complaint represents the reason for the patient’s visit.…”

Section: Related Workmentioning

confidence: 99%

A Disease-Prediction Protocol Integrating Triage Priority and BERT-Based Transfer Learning for Intelligent Triage

et al. 2023

View full text Add to dashboard Cite

Large hospitals can be complex, with numerous discipline and subspecialty settings. Patients may have limited medical knowledge, making it difficult for them to determine which department to visit. As a result, visits to the wrong departments and unnecessary appointments are common. To address this issue, modern hospitals require a remote system capable of performing intelligent triage, enabling patients to perform self-service triage. To address the challenges outlined above, this study presents an intelligent triage system based on transfer learning, capable of processing multilabel neurological medical texts. The system predicts a diagnosis and corresponding department based on the patient’s input. It utilizes the triage priority (TP) method to label diagnostic combinations found in medical records, converting a multilabel problem into a single-label one. The system considers disease severity and reduces the “class overlapping” of the dataset. The BERT model classifies the chief complaint text, predicting a primary diagnosis corresponding to the complaint. To address data imbalance, a composite loss function based on cost-sensitive learning is added to the BERT architecture. The study results indicate that the TP method achieves a classification accuracy of 87.47% on medical record text, outperforming other problem transformation methods. By incorporating the composite loss function, the system’s accuracy rate improves to 88.38% surpassing other loss functions. Compared to traditional methods, this system does not introduce significant complexity, yet substantially improves triage accuracy, reduces patient input confusion, and enhances hospital triage capabilities, ultimately improving the patient’s medical experience. The findings could provide a reference for intelligent triage development.

show abstract

Section: Related Workmentioning

confidence: 99%

A Disease-Prediction Protocol Integrating Triage Priority and BERT-Based Transfer Learning for Intelligent Triage

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Bidirectional Encoder Representations from Transformers (BERT). We selected general domain RuBERT (12 layers, 12 self-attention heads, and 768 hidden layer size) [11] as the base model for evaluation with the transformer models and RuPoolBERT [2] as its extended version. Both models fine-tuned for each task with the fixed set of hyperparameters, e.g., an input sequence length is 256 tokens, 25 training epochs with a learning rate of 3 × 10 −5 .…”

Section: Baselinesmentioning

confidence: 99%

RuMedBench: A Russian Medical Language Understanding Benchmark

Blinov,

Reshetnikova,

Nesterov

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The paper describes the open Russian medical language understanding benchmark covering several task types (classification, question answering, natural language inference, named entity recognition) on a number of novel text sets. Given the sensitive nature of the data in healthcare, such a benchmark partially closes the problem of Russian medical dataset absence. We prepare the unified format labeling, data split, and evaluation metrics for new tasks. The remaining tasks are from existing datasets with a few modifications. A single-number metric expresses a model's ability to cope with the benchmark. Moreover, we implement several baseline models, from simple ones to neural networks with transformer architecture, and release the code. Expectedly, the more advanced models yield better performance, but even a simple model is enough for a decent result in some tasks. Furthermore, for all tasks, we provide a human evaluation. Interestingly the models outperform humans in the large-scale classification tasks. However, the advantage of natural intelligence remains in the tasks requiring more knowledge and reasoning.

show abstract

“…The version of the model with CLS representation as the hidden state (v.CLS ) is actually performing poorly. But the MusaNet 's result is easily overcome by using more advanced pooling strategies over contextualized embeddings proposed in [3]. Such a model (v.cmm wo gender/age) achieves comparable results (given the standard deviation) even without gender and age embeddings.…”

Section: Mimic-iii Benchmarkmentioning

confidence: 99%

Patient Embeddings in Healthcare and Insurance Applications

Blinov¹,

Kokh²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The paper researches the problem of concept and patient representations in the medical domain. We present the patient histories from Electronic Health Records (EHRs) as temporal sequences of ICD concepts for which embeddings are learned in an unsupervised setup with a transformer-based neural network model. The model training was performed on the collection of one million patients' histories in 6 years. The predictive power of such a model is assessed in comparison with several baseline methods. A series of experiments on the MIMIC-III data show the advantage of the presented model compared to a similar system. Further, we analyze the obtained embedding space with regards to concept relations and show how knowledge from the medical domain can be successfully transferred to the practical task of insurance scoring in the form of patient embeddings.

show abstract

Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-Based Neural Networks

Cited by 17 publications

References 13 publications

A Disease-Prediction Protocol Integrating Triage Priority and BERT-Based Transfer Learning for Intelligent Triage

A Disease-Prediction Protocol Integrating Triage Priority and BERT-Based Transfer Learning for Intelligent Triage

RuMedBench: A Russian Medical Language Understanding Benchmark

Patient Embeddings in Healthcare and Insurance Applications

Contact Info

Product

Resources

About