Multi-label classification for biomedical literature: an overview of the BioCreative VII LitCovid Track for COVID-19 literature topic annotations

Chen, Qingyu; Allot, Alexis; Leaman, Robert; Doğan, Rezarta Islamaj; Du, Jingcheng; Li, Fang; Wang, Kai; Xu, Su; Zhang, Yuefu; Bagherzadeh, Parsa; Bergler, Sabine; Bhatnagar, Aakash; Bhavsar, Nidhir; Chang, Yuan‐Chieh; Lin, Sheng-Jie; Tang, Wentai; Zhang, Hongtong; Ilija, Tavchioski,; Pollak, Senja; Tian, Shubo; Zhang, Jinfeng; Otmakhova, Yulia; Yepes, Antonio Jimeno; Dong, Hang; Wu, Honghan; Dufour, Richard; Labrak, Yanis; Chatterjee, Niladri; Tandon, Kushagri; Laleye, Fréjus A. A.; Rakotoson, Loïc; Chersoni, Emmanuele; Gu, Jinghang; Friedrich, Annemarie; Pujari, Subhash Chandra; Chizhikova, Mariia; Sivadasan, Naveen; Lu, Zhiyong

doi:10.48550/arxiv.2204.09781

Cited by 3 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspired by previous research [ 26 , 30 , 34 ], this article is devoted to the COVID-19 semantic indexing problem. Our goal is to develop a benchmark dataset and a robust yet flexible semantic topic identification framework for the COVID-19 domain, which has not been addressed in previous research.…”

Section: Related Workmentioning

confidence: 75%

“…In recent decades, to facilitate the research of biomedical topic curation, a series of automated methods [ 22 – 32 ] and challenging competitions [ 33 , 34 ] have been developed to improve the time-consuming, costly, and labor-intensive semantic indexing process.…”

Section: Related Workmentioning

confidence: 99%

“…Recently, in response to the worldwide pandemic, the focus of research has drastically shifted towards the specific concepts and sub-concepts of coronavirus. The BioCreative-VII community proposed the challenging task of the LitCovid Track [ 34 ], which targets identifying semantic topics to the COVID-19 relevant literature. Accordingly, the LitCovid task is regarded as a multilabel classification problem and engaged worldwide efforts to provide practical benefits to the COVID-19 topic curation.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Multi-probe attention neural network for COVID-19 semantic indexing

Xiang

Wang

et al. 2022

BMC Bioinformatics

Self Cite

View full text Add to dashboard Cite

Background The COVID-19 pandemic has increasingly accelerated the publication pace of scientific literature. How to efficiently curate and index this large amount of biomedical literature under the current crisis is of great importance. Previous literature indexing is mainly performed by human experts using Medical Subject Headings (MeSH), which is labor-intensive and time-consuming. Therefore, to alleviate the expensive time consumption and monetary cost, there is an urgent need for automatic semantic indexing technologies for the emerging COVID-19 domain. Results In this research, to investigate the semantic indexing problem for COVID-19, we first construct the new COVID-19 Semantic Indexing dataset, which consists of more than 80 thousand biomedical articles. We then propose a novel semantic indexing framework based on the multi-probe attention neural network (MPANN) to address the COVID-19 semantic indexing problem. Specifically, we employ a k-nearest neighbour based MeSH masking approach to generate candidate topic terms for each input article. We encode and feed the selected candidate terms as well as other contextual information as probes into the downstream attention-based neural network. Each semantic probe carries specific aspects of biomedical knowledge and provides informatively discriminative features for the input article. After extracting the semantic features at both term-level and document-level through the attention-based neural network, MPANN adopts a linear multi-view classifier to conduct the final topic prediction for COVID-19 semantic indexing. Conclusion The experimental results suggest that MPANN promises to represent the semantic features of biomedical texts and is effective in predicting semantic topics for COVID-19 related biomedical articles.

show abstract

Section: Related Workmentioning

confidence: 75%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Multi-probe attention neural network for COVID-19 semantic indexing

Xiang

Wang

et al. 2022

BMC Bioinformatics

Self Cite

View full text Add to dashboard Cite

show abstract

“…Extensive studies have shown the superiority of transformer-based deep learning models for many NLP tasks [ 11 , 13 – 16 ]. Based on our experiments, however, adding features to the pre-trained language models that have not seen these features before may not significantly boost their performance.…”

Section: Discussionmentioning

confidence: 99%

Machine learning models for abstract screening task - A systematic literature review application for health economics and outcome research

Du,

Soysal,

Wang

et al. 2024

BMC Med Res Methodol

Self Cite

View full text Add to dashboard Cite

Objective Systematic literature reviews (SLRs) are critical for life-science research. However, the manual selection and retrieval of relevant publications can be a time-consuming process. This study aims to (1) develop two disease-specific annotated corpora, one for human papillomavirus (HPV) associated diseases and the other for pneumococcal-associated pediatric diseases (PAPD), and (2) optimize machine- and deep-learning models to facilitate automation of the SLR abstract screening. Methods This study constructed two disease-specific SLR screening corpora for HPV and PAPD, which contained citation metadata and corresponding abstracts. Performance was evaluated using precision, recall, accuracy, and F1-score of multiple combinations of machine- and deep-learning algorithms and features such as keywords and MeSH terms. Results and conclusions The HPV corpus contained 1697 entries, with 538 relevant and 1159 irrelevant articles. The PAPD corpus included 2865 entries, with 711 relevant and 2154 irrelevant articles. Adding additional features beyond title and abstract improved the performance (measured in Accuracy) of machine learning models by 3% for HPV corpus and 2% for PAPD corpus. Transformer-based deep learning models that consistently outperformed conventional machine learning algorithms, highlighting the strength of domain-specific pre-trained language models for SLR abstract screening. This study provides a foundation for the development of more intelligent SLR systems.

show abstract

Multi-label classification for biomedical literature: an overview of the BioCreative VII LitCovid Track for COVID-19 literature topic annotations

et al. 2022

View full text Add to dashboard Cite

The coronavirus disease 2019 (COVID-19) pandemic has been severely impacting global society since December 2019. The related findings such as vaccine and drug development have been reported in biomedical literature—at a rate of about 10 000 articles on COVID-19 per month. Such rapid growth significantly challenges manual curation and interpretation. For instance, LitCovid is a literature database of COVID-19-related articles in PubMed, which has accumulated more than 200 000 articles with millions of accesses each month by users worldwide. One primary curation task is to assign up to eight topics (e.g. Diagnosis and Treatment) to the articles in LitCovid. The annotated topics have been widely used for navigating the COVID literature, rapidly locating articles of interest and other downstream studies. However, annotating the topics has been the bottleneck of manual curation. Despite the continuing advances in biomedical text-mining methods, few have been dedicated to topic annotations in COVID-19 literature. To close the gap, we organized the BioCreative LitCovid track to call for a community effort to tackle automated topic annotation for COVID-19 literature. The BioCreative LitCovid dataset—consisting of over 30 000 articles with manually reviewed topics—was created for training and testing. It is one of the largest multi-label classification datasets in biomedical scientific literature. Nineteen teams worldwide participated and made 80 submissions in total. Most teams used hybrid systems based on transformers. The highest performing submissions achieved 0.8875, 0.9181 and 0.9394 for macro-F1-score, micro-F1-score and instance-based F1-score, respectively. Notably, these scores are substantially higher (e.g. 12%, higher for macro F1-score) than the corresponding scores of the state-of-art multi-label classification method. The level of participation and results demonstrate a successful track and help close the gap between dataset curation and method development. The dataset is publicly available via https://ftp.ncbi.nlm.nih.gov/pub/lu/LitCovid/biocreative/ for benchmarking and further development. Database URL https://ftp.ncbi.nlm.nih.gov/pub/lu/LitCovid/biocreative/

show abstract

Multi-label classification for biomedical literature: an overview of the BioCreative VII LitCovid Track for COVID-19 literature topic annotations

Cited by 3 publications

References 0 publications

Multi-probe attention neural network for COVID-19 semantic indexing

Multi-probe attention neural network for COVID-19 semantic indexing

Machine learning models for abstract screening task - A systematic literature review application for health economics and outcome research

Multi-label classification for biomedical literature: an overview of the BioCreative VII LitCovid Track for COVID-19 literature topic annotations

Contact Info

Product

Resources

About