Multi-label classification of research articles using Word2Vec and identification of similarity threshold

Abstract

Every year, around 28,100 journals publish 2.5 million research publications. Search engines, digital libraries, and citation indexes are used extensively to search these publications. When a user submits a query, it generates a large number of documents among which just a few are relevant. Due to inadequate indexing, the resultant documents are largely unstructured. Publicly known systems mostly index the research papers using keywords rather than using subject hierarchy. Numerous methods reported for performing single-label classification (SLC) or multi-label classification (MLC) are based on content and metadata features. Content-based techniques offer higher outcomes due to the extreme richness of features. But the drawback of content-based techniques is the unavailability of full text in most cases. The use of metadata-based parameters, such as title, keywords, and general terms, acts as an alternative to content. However, existing metadata-based techniques indicate low accuracy due to the use of traditional statistical measures to express textual properties in quantitative form, such as BOW, TF, and TFIDF. These measures may not establish the semantic context of the words. The existing MLC techniques require a specified threshold value to map articles into predetermined categories for which domain knowledge is necessary. The objective of this paper is to get over the limitations of SLC and MLC techniques. To capture the semantic and contextual information of words, the suggested approach leverages the Word2Vec paradigm for textual representation. The suggested model determines threshold values using rigorous data analysis, obviating the necessity for domain expertise. Experimentation is carried out on two datasets from the field of computer science (JUCS and ACM). In comparison to current state-of-the-art methodologies, the proposed model performed well. Experiments yielded average accuracy of 0.86 and 0.84 for JUCS and ACM for SLC, and 0.81 and 0.80 for JUCS and ACM for MLC. On both datasets, the proposed SLC model improved the accuracy up to 4%, while the proposed MLC model increased the accuracy up to 3%.

Prediction of 35 Target Per- and Polyfluoroalkyl Substances (PFASs) in California Groundwater Using Multilabel Semisupervised Machine Learning

“…The smaller the hamming loss, the better the model performance . We also used average precision and average recall, where the partially correct concept is considered to calculate the average for all the samples. , We used the Exact Match Accuracy, where the result would be considered correct when the predicted set of labels exactly matches the true label for each sample. , We also calculated the AUROC score for each PFAS and calculated the average AUROC for each multilabel model (equation in SI Table S6). The development and evaluation of ML models were coded with the sklearn, xgboost, CatBoost, lightgbm, and PyTorch (TabNet) packages in Python.…”

“…42 We also used average precision and average recall, where the partially correct concept is considered to calculate the average for all the samples. 43,44 We used the Exact Match Accuracy, where the result would be considered correct when the predicted set of labels exactly matches the true label for each sample. 45,42 We also calculated the AUROC score for each PFAS and calculated the average AUROC for each multilabel model (equation in SI Table S6).…”

A Comparison of Multi-Label Text Classification Models in Research Articles Labeled With Sustainable Development Goals

“…Once documents are tokenized, a text feature extraction method is applied to obtain the most distinguishing features of a text, reducing dimensionality [35]- [38]. Some of the broadly feature extraction techniques in research article classification approaches are: 1) One Hot Encoding, 2) Bag of Word (BOW) or Term Frequency (TF), and 3) Term Frequency and Inverse Document Frequency (TF-IDF), and semantic based approaches are: 1) Glove, 2) FastText, and 3) Word2Vec [24].…”

Exploiting Label Dependencies for Multi-Label Document Classification Using Transformers

“…In multi-label text classification, the goal is to associate one or more labels to the input text. It is an important task that has applications in many tasks such as research article classification and metadata generation from documents [Mustafa et al 2021;Sajid et al 2011] that can be used for optimizing search engine indexing.…”