BackgroundThe aim of this study was to validate a three-class sentiment classification model for clinical trial abstracts combining adversarial learning and the BioBERT language processing model as a tool to assess trends in biomedical literature in a clearly reproducible manner. We then assessed the model's performance for this application and compared it to previous models used for this task.MethodsUsing 108 expert-annotated clinical trial abstracts and 2,000 unlabeled abstracts this study develops a three-class sentiment classification algorithm for clinical trial abstracts. The model uses a semi-supervised model based on the Bidirectional Encoder Representation from Transformers (BERT) model, a much more advanced and accurate method compared to previously used models based upon traditional machine learning methods. The prediction performance was compared to those previous studies.ResultsThe algorithm was found to have a classification accuracy of 91.3%, with a macro F1-Score of 0.92, significantly outperforming previous studies used to classify sentiment in clinical trial literature, while also making the sentiment classification finer grained with greater reproducibility.ConclusionWe demonstrate an easily applied sentiment classification model for clinical trial abstracts that significantly outperforms previous models with greater reproducibility and applicability to large-scale study of reporting trends.
IntroductionSentiment analysis, by evaluating written wording and its context, is a growing tool used in computer science that can determine the level of support expressed in a body of text using artificial intelligence methodologies. The application of sentiment analysis to biomedical literature is a growing field and offers the potential to rapidly and economically explore large amounts of published research and characterize treatment efficacy.MethodsWe compared the results of sentiment analysis of 115 article abstracts analyzed in a recently published meta-analysis of peripheral nerve block usage in primary hip and knee arthroplasty to the conclusions drawn by the authors of the original meta-analysis.ResultsA moderately positive outlook supporting the utilization of regional anesthesia for hip and knee arthroplasty was found in the 115 articles that were included for analysis, with 46% expressing positive sentiment, 35% expressing neutral sentiment, and 19% of abstracts expressing negative sentiment. This was well aligned with the conclusions reached by a previous meta-analysis of the same articles.DiscussionSentiment analysis applied to the medical literature can rapidly evaluate large collections of published data and generate an impression of overall findings that are aligned with the findings of a traditional meta-analysis.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.