HITSZ-ICRC: A Report for SMM4H Shared Task 2019-Automatic Classification and Extraction of Adverse Effect Mentions in Tweets

Abstract

This is the system description of the Harbin Institute of Technology Shenzhen (HITSZ) team for the first and second subtasks of the fourth Social Media Mining for Health Applications (SMM4H) shared task in 2019. The two subtasks are automatic classification and extraction of adverse effect mentions in tweets. The systems for the two subtasks are based on bidirectional encoder representations from transformers (BERT), and achieves promising results. Among the systems we developed for subtask1, the best F1-score was 0.6457, for subtask2, the best relaxed F1-score and the best strict F1-score were 0.614 and 0.407 respectively. Our system ranks first among all systems on subtask1.

BERT Goes Brrr: A Venture Towards the Lesser Error in Classifying Medical Self-Reporters on Twitter

“…Motivated by some past successful results (Chen et al, 2019;Casola and Lavelli, 2020), we ensembled some trained models on Task 1a and Task 5, which are picked from the best performing HPO models. In the implementation of the ensemble technique, to predict the label of an instance, we summed all of the chosen models' probability score and took the highest score as the label.…”

An Effective Emotional Expression and Knowledge-Enhanced Method for Detecting Adverse Drug Reactions

“…Transfer learning [8], co-training [9] and multi-task learning [39] are adopted to extract ADRs, classify tweets mentioning ADRs and normalize ADRs concept, and multitask learning achieves the state-of-the-art result. With BERT performing well in many NLP tasks, researchers introduce the knowledge base and conditional random field (CRF) into BERT for the automatic classification of ADRs (text classification) and extraction of ADRs (NER) on SMM4H Shared Task 2019 [19], respectively.…”

“…In our experiments, two Twitter corpora are employed to verify the effectiveness of the proposed model and perform comparison experiments of the baselines. The twitter ADR dataset from PSB2016-Task1 [4] contains 10,822 tweets in the original dataset, while the dataset from Social Media Mining for Health Applications (SMM4H) Workshop & Shared Task 2018-Task3 [19], which is also an extension of the PSB2016-Task1 dataset, consists of approximately 15,000 annotated tweets as training data and 9000 annotated test tweets, respectively. These tweets related to drugs prescribed for chronic diseases and the prevalence of drug use were annotated by two domain experts under the guidance of a pharmacology expert [2].…”

“…This is a state-of-the-art model for performing the ADR classification task on SMM4H [19]. The model builds <drug, ADR> pairs, generates binary features and then integrates the features with the output of BERT.…”

“…Adverse event entities have been extracted from patient forums [2] using drug safety databases [18] such as MedEffect and COSTART. Recent studies have also adopted MedDRA and SIDER to better understand and match users' expressions of drugs and ADRs in social media [19]. These methods only utilize the medical resource to supplement features rather than enhance information representation for short social texts, but they overlook the characteristics of insufficient information representation due to text length limitation.…”