An Improved Double Channel Long Short-Term Memory Model for Medical Text Classification

Liang, Shengbin; Chen, Xinan; Ma, Jixin; Wei, Du; Ma, Huawei

doi:10.1155/2021/6664893

Cited by 8 publications

(5 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the training speed of the CBOW model is slow. Inspired by the FastText [36] model, some scholars proposed some models such as CNN-LSTM [37] and DC-LSTM [38], and they used FastText to calculate word vectors and used LSTM model to solve the difficulty of information transmission caused by long sequence data. Edara et al [31] analyzed moods of various cancer affected patients by collecting tweets from different online cancer-supported communities.…”

Section: Plos Onementioning

confidence: 99%

A model of integrating convolution and BiGRU dual-channel mechanism for Chinese medical text classifications

Zhang

Jin

et al. 2023

PLoS ONE

Self Cite

View full text Add to dashboard Cite

Recently, a lot of Chinese patients consult treatment plans through social networking platforms, but the Chinese medical text contains rich information, including a large number of medical nomenclatures and symptom descriptions. How to build an intelligence model to automatically classify the text information consulted by patients and recommend the correct department for patients is very important. In order to address the problem of insufficient feature extraction from Chinese medical text and low accuracy, this paper proposes a dual channel Chinese medical text classification model. The model extracts feature of Chinese medical text at different granularity, comprehensively and accurately obtains effective feature information, and finally recommends departments for patients according to text classification. One channel of the model focuses on medical nomenclatures, symptoms and other words related to hospital departments, gives different weights, calculates corresponding feature vectors with convolution kernels of different sizes, and then obtains local text representation. The other channel uses the BiGRU network and attention mechanism to obtain text representation, highlighting the important information of the whole sentence, that is, global text representation. Finally, the model uses full connection layer to combine the representation vectors of the two channels, and uses Softmax classifier for classification. The experimental results show that the accuracy, recall and F1-score of the model are improved by 10.65%, 8.94% and 11.62% respectively compared with the baseline models in average, which proves that our model has better performance and robustness.

show abstract

Section: Plos Onementioning

confidence: 99%

A model of integrating convolution and BiGRU dual-channel mechanism for Chinese medical text classifications

Zhang

Jin

et al. 2023

PLoS ONE

Self Cite

View full text Add to dashboard Cite

show abstract

“…Venkataraman et al 31 trained an RNN to automate clinical record assignment. Liang et al 32 improved LSTM with a dual-channel mechanism for Chinese medical text classification. Bangyal et al 33 applied RNN and LSTM to COVID-19 fake news detection, a medical text classification task.…”

Section: Related Workmentioning

confidence: 99%

Medical text classification based on the discriminative pre-training model and prompt-tuning

Wang

Peng

et al. 2023

DIGITAL HEALTH

View full text Add to dashboard Cite

Medical text classification, as a fundamental medical natural language processing task, aims to identify the categories to which a short medical text belongs. Current research has focused on performing the medical text classification task using a pre-training language model through fine-tuning. However, this paradigm introduces additional parameters when training extra classifiers. Recent studies have shown that the “prompt-tuning” paradigm induces better performance in many natural language processing tasks because it bridges the gap between pre-training goals and downstream tasks. The main idea of prompt-tuning is to transform binary or multi-classification tasks into mask prediction tasks by fully exploiting the features learned by pre-training language models. This study explores, for the first time, how to classify medical texts using a discriminative pre-training language model called ERNIE-Health through prompt-tuning. Specifically, we attempt to perform prompt-tuning based on the multi-token selection task, which is a pre-training task of ERNIE-Health. The raw text is wrapped into a new sequence with a template in which the category label is replaced by a [UNK] token. The model is then trained to calculate the probability distribution of the candidate categories. Our method is tested on the KUAKE-Question Intention Classification and CHiP-Clinical Trial Criterion datasets and obtains the accuracy values of 0.866 and 0.861. In addition, the loss values of our model decrease faster throughout the training period compared to the fine-tuning. The experimental results provide valuable insights to the community and suggest that prompt-tuning can be a promising approach to improve the performance of pre-training models in domain-specific tasks.

show abstract

“…A two-channel LSTM method that is commonly used in areas such as text classification (Liang et al 2021) and that uses both the Niño-3.4 index and SC-PRECIP as inputs to predict both of the indices and to capture the nonlinear shared variability of the two indices also performs poorly in comparison with the MIMO-AE index (Fig. 10).…”

Section: Sc-precip Predictability From Mimo-ae Indexmentioning

confidence: 99%

“…Multitask learning, on the other hand, solves multiple learning tasks at the same time while exploiting commonalities and differences across tasks (Caruana 1997). It has been applied to many problems including natural language processing (Chen et al 2021), speech recognition (Toshniwal et al 2017), and computer vision (Cipolla et al 2018) to improve the prediction accuracy and learning efficiency of task-specific models. Ghifary et al (2015) implemented multitask learning with a multioutput autoencoder, for related cross-domain object recognition.…”

Section: Introductionmentioning

confidence: 99%

Assessing Tropical Pacific–Induced Predictability of Southern California Precipitation Using a Novel Multi-Input Multioutput Autoencoder

Passarella

Mahajan

2023

Artificial Intelligence for the Earth Systems

View full text Add to dashboard Cite

We construct a novel Multi-Input Multi-Output Autoencoder-decoder (MIMO-AE) to capture the non-linear relationship of Southern California precipitation and tropical Pacific Ocean sea surface temperature. The MIMO-AE is trained on both monthly TP-SST and SC-PRECIP anomalies simultaneously. The co-variability of the two fields in the MIMO-AE shared nonlinear latent space can be condensed into an index, termed the MIMO-AE index. We use a transfer learning approach to train a MIMO-AE on the combined dataset of 100 years of output from a historical simulation with the Energy Exascale Earth Systems Model version 1 and a segment of observational data. We further use Long Short-Term Memory networks to assess sub-seasonal predictability of SC-PRECIP using the MIMO-AE index. We find that the MIMO-AE index provides enhanced predictability of SC-PRECIP for a lead-time of up-to four months as compared to Niño 3.4 index and the El Niño Southern Oscillation Longitudinal Index.

show abstract

An Improved Double Channel Long Short-Term Memory Model for Medical Text Classification

Cited by 8 publications

References 16 publications

A model of integrating convolution and BiGRU dual-channel mechanism for Chinese medical text classifications

A model of integrating convolution and BiGRU dual-channel mechanism for Chinese medical text classifications

Medical text classification based on the discriminative pre-training model and prompt-tuning

Assessing Tropical Pacific–Induced Predictability of Southern California Precipitation Using a Novel Multi-Input Multioutput Autoencoder

Contact Info

Product

Resources

About