View Distillation with Unlabeled Data for Extracting Adverse Drug Effects from User-Generated Data

Abstract

We present an algorithm based on multi-layer transformers for identifying Adverse Drug Reactions (ADR) in social media data. Our model relies on the properties of the problem and the characteristics of contextual word embeddings to extract two views from documents. Then a classifier is trained on each view to label a set of unlabeled documents to be used as an initializer for a new classifier in the other view. Finally, the initialized classifier in each view is further trained using the initial training examples. We evaluated our model in the largest publicly available ADR dataset. The experiments testify that our model significantly outperforms the transformer-based models pretrained on domain-specific data.

Multiple-Source Domain Adaptation via Coordinated Domain Encoders and Paired Classifiers

“…Modern classifiers typically rely on large amount of training data. Collecting large training sets is expensive and in some cases very challenging, e.g., in legal domain (Holzenberger, Blair-Stanek, and Van Durme 2020) or in social media domain (Karisani, Choi, and Xiong 2021;Karisani and Karisani 2020). There exist several techniques to address the lack of training data, one of which is Domain Adaptation (Ben-David and Schuller 2003), where a classifier is trained in one domain (the source domain) and evaluated in another domain (the target domain).…”