Document Classification for COVID-19 Literature

Gutiérrez, Bernal Jiménez; Zeng, Juncheng; Zhang, Dongdong; Zhang, Ping; Su, Yu

doi:10.48550/arxiv.2006.13816

Cited by 3 publications

(7 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The observations are similar when comparing LITMC-BERT with Linear BERT: e.g., its macro-F1 and accuracy are up to 2% and 4% higher on the HoC dataset, respectively. In terms of comparing Binary BERT with Linear BERT, Binary BERT achieved overall better performance on the LitCovid Bi-oCreative dataset, which is consistent with the literature [6,15], whereas Linear BERT achieved over better performance on the HoC dataset.…”

Section: Statistic Test and Reporting Standardsupporting

confidence: 86%

“…Such methods have achieved promising performance in a range of multilabel classification tasks [13,22]. Indeed, existing studies have shown binary relevance BERT achieved the best performance for topic annotation in LitCovid [6,15]. However, it is computationally expensive and transforming multi-label classification tasks into binary classification may ignore the correlations among labels.…”

Section: Multi-label Classificationmentioning

confidence: 99%

“…Note that previous studies mostly have used Binary BERT in biomedical literature [16,17]. It also has been the state-of-the-art (SOTA) model for multi-label classification in biomedical literature [15][16][17] and was also used in the LitCovid production system previously [6].…”

Section: Baseline Modelsmentioning

confidence: 99%

“…To facilitate manual topic annotation, we previously employed the deep learning model Bidirectional Encoder Representations from Transformers (BERT) [14]. We used one BERT model per topic, known as Binary BERT, and previously demonstrated this method achieved the best performance of the available models for LitCovid topic annotations [6]; other studies have reported consistent results [15]. Indeed, existing studies in biomedical text mining have demonstrated Binary BERT achieves the best performance in multi-label classification tasks [16,17].…”

Section: Introductionmentioning

confidence: 95%

See 3 more Smart Citations

LitMC-BERT: Transformer-Based Multi-Label Classification of Biomedical Literature With An Application on COVID-19 Literature Curation

Chen

Du²,

Allot

et al. 2022

IEEE/ACM Trans. Comput. Biol. and Bioinf.

View full text Add to dashboard Cite

The rapid growth of biomedical literature poses a significant challenge for curation and interpretation. This has become more evident during the COVID-19 pandemic. LitCovid, a literature database of COVID-19 related papers in PubMed, has accumulated over 180,000 articles with millions of accesses. Approximately 10,000 new articles are added to LitCovid every month. A main curation task in LitCovid is topic annotation where an article is assigned with up to eight topics, e.g., Treatment and Diagnosis. The annotated topics have been widely used both in LitCovid (e.g., accounting for ~18% of total uses) and downstream studies such as network generation. However, it has been a primary curation bottleneck due to the nature of the task and the rapid literature growth. This study proposes LITMC-BERT, a transformer-based multi-label classification method in biomedical literature. It uses a shared transformer backbone for all the labels while also captures label-specific features and the correlations between label pairs. We compare LITMC-BERT with three baseline models on two datasets. Its micro-F1 and instance-based F1 are 5% and 4% higher than the current best results, respectively, and only requires ~18% of the inference time than the Binary BERT baseline. The related datasets and models are available via https://github.com/ncbi/ml-transformer.

show abstract

Section: Statistic Test and Reporting Standardsupporting

confidence: 86%

Section: Multi-label Classificationmentioning

confidence: 99%

Section: Baseline Modelsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 95%

See 2 more Smart Citations

LitMC-BERT: Transformer-Based Multi-Label Classification of Biomedical Literature With An Application on COVID-19 Literature Curation

Chen

Du²,

Allot

et al. 2022

IEEE/ACM Trans. Comput. Biol. and Bioinf.

View full text Add to dashboard Cite

show abstract

“…Future studies may involve changing the SciBERT model to a There are a few previous studies that have the same objective as our study, classified in the CORD-19 dataset. A study from 2020 [51] used the LitCovid dataset to train the machine learning model. In this article, they trained Logistic Regression and Support Vector Machine for the traditional machine learning model, LSTM model for the neural network model and fine-tuned the BioBERT, Longformer [52] for the BERT based model.…”

Section: Discussionmentioning

confidence: 99%

Do We Need a Specific Corpus and Multiple High-Performance GPUs for Training the BERT Model? An Experiment on COVID-19 Dataset

Nuntachit

Sugunnasil

2022

MAKE

View full text Add to dashboard Cite

The COVID-19 pandemic has impacted daily lives around the globe. Since 2019, the amount of literature focusing on COVID-19 has risen exponentially. However, it is almost impossible for humans to read all of the studies and classify them. This article proposes a method of making an unsupervised model called a zero-shot classification model, based on the pre-trained BERT model. We used the CORD-19 dataset in conjunction with the LitCovid database to construct new vocabulary and prepare the test dataset. For NLI downstream task, we used three corpora: SNLI, MultiNLI, and MedNLI. We significantly reduced the training time by 98.2639% to build a task-specific machine learning model, using only one Nvidia Tesla V100. The final model can run faster and use fewer resources than its comparators. It has an accuracy of 27.84%, which is lower than the best-achieved accuracy by 6.73%, but it is comparable. Finally, we identified that the tokenizer and vocabulary more specific to COVID-19 could not outperform the generalized ones. Additionally, it was found that BART architecture affects the classification results.

show abstract

A Practical and Empirical Comparison of Three Topic Modeling Methods Using a COVID-19 Corpus: LSA, LDA, and Top2Vec

Zengul,

Bulut,

Oner

et al. 2023

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

This study was prepared as a practical guide for researchers interested in using topic modeling methodologies. This study is specially designed for those with difficulty determining which methodology to use. Many topic modeling methods have been developed since the 1980s namely, latent semantic indexing or analysis (LSI/LSA), , probabilistic LSI/LSA (pLSI/pLSA), naïve Bayes, the Author-Recipient-Topic (ART), Latent Dirichlet Allocation (LDA), Topic Over Time (TOT), Dynamic Topic Models (DTM), Word2Vec, Top2Vec and \variation and combination of these techniques. For researchers from disciplines other than computer science may find it challenging to select a topic modeling methodology. We compared a recently developed topic modeling algorithm Top2Vec with two of the most conventional and frequently-used methodologiesLSA and LDA. As a study sample, we used a corpus of 65,292 COVID-19-focused abstracts. Among the 11 topics we identified in each methodology, we found high levels of correlation between LDA and Top2Vec results, followed by LSA and LDA and Top2Vec and LSA. We also provided information on computational resources we used to perform the analyses and provided practical guidelines and recommendations for researchers.

show abstract

Document Classification for COVID-19 Literature

Cited by 3 publications

References 8 publications

LitMC-BERT: Transformer-Based Multi-Label Classification of Biomedical Literature With An Application on COVID-19 Literature Curation

LitMC-BERT: Transformer-Based Multi-Label Classification of Biomedical Literature With An Application on COVID-19 Literature Curation

Do We Need a Specific Corpus and Multiple High-Performance GPUs for Training the BERT Model? An Experiment on COVID-19 Dataset

A Practical and Empirical Comparison of Three Topic Modeling Methods Using a COVID-19 Corpus: LSA, LDA, and Top2Vec

Contact Info

Product

Resources

About