A deep learning classifier for sentence classification in biomedical and computer science abstracts

Abstract

The automatic classification of abstract sentences into its main elements (background, objectives, methods, results, conclusions) is a key tool to support scientific database querying, to summarize relevant literature works and to assist in the writing of new abstracts. In this paper, we propose a novel deep learning approach based on a convolutional layer and a bi-directional gated recurrent unit to classify sentences of abstracts. First, the proposed neural network was tested on a publicly available repository containing 20 thousand abstracts from the biomedical domain. Competitive results were achieved, with weight-averaged precision, recall and F1-score values around 91%, and an Area Under the ROC Curve (AUC) of 99%, which are higher when compared to a state-of-the-art neural network. Then, a crowdsourcing approach using gamification was adopted to create a new comprehensive set of 4,111 classified sentences from the computer science domain, focused on social media abstracts. The results of applying the same deep learning modeling technique trained with 3,287 (80%) of the available sentences were below the ones obtained for the larger biomedical dataset, with weight-averaged precision, recall and F1-score values between 73% and 76%, and an AUC of 91%. Considering the dataset dimension as a likely important factor for such performance decrease, a data augmentation approach was further applied. This involved the use of text mining to translate sentences of the computer science abstract corpus while retaining the same meaning. Such approach resulted in slight improvements, of around 2 percentage points, for the weight-averaged recall and F1-score values.

Twitter alloy steel disambiguation and user relevance via one-class and two-class news titles classifiers

“…Then, we apply the non-parametric Wilcoxon test for measuring statistical significance [62]. To compare the different classifiers, we use the popular area under the curve (AUC) of the receiver operating characteristic (ROC) curve [45,63,64], computed on the rolling window test data. The ROC curve shows the performance of a classifier for a target class and across all decision threshold values (T TFD and T FUR ), plotting the False Positive Rate (FPR), in x-axis, versus the True Positive Rate (TPR), in the y-axis.…”

“…The AU C = ROCdT measures the global discriminatory performance of a classifier. Often, the AUC values are interpreted as [64]: 0.5equal to a random classifier; 0.6 -reasonable, 0.7 -good; 0.8 -very good; 0.9excellent; and 1 -perfect. The ROC curve analysis contains two main advantages to evaluate classifiers [63].…”

Algerian Dialect Translation Applied on COVID-19 Social Media Comments

“…In the last number of scholarly works, there have been remarkable developments in deep learning [ 12 , 13 ]. Architectures such as a convolutional neural network (CNN), LSTM, and GRU have obtained competitive results in several competitions (e.g., computer vision, signal, and natural language processing) [ 14 ]. The Long Short-Term Memory networks “LSTMs”, introduced in [ 15 ], are a special kind of RNN, capable of learning long-term dependencies.…”

Toward Building an Academic Search Engine Understanding the Purposes of the Matched Sentences in an Abstract

“…However, such a design assumes that the class of the current sentence is conditionally independent of the classes of the future sentences and the previous n (n ≥ 2) sentences given the class of the previous sentence. The other model, Word-BiGRU [10] employs word embeddings within the same sentence to generate sentence embeddings, which are further used to label the class of the sentence. The Word-BiGRU model utilizes convolution layers with filter sizes of 5 to integrate the words within a sentence.…”

“…The fifth model, Word-BiGRU [10], also generates each sentence embedding by integrating the word embeddings within each sentence. This model incorporates the relationship among different sentences by a bidirectional gated recurrent unit (GRU).…”