Recognizing Continuous and Discontinuous Adverse Drug Reaction Mentions from Social Media Using LSTM‐CRF

Tang, Buzhou; Hu, Jianglu; Wang, Xiaolong; Chen, Qingcai

doi:10.1155/2018/2379208

Cited by 37 publications

(31 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nonetheless, the vast majority of systems developed using these two datasets focused on one single subtask, the location of the ADR mention for systems using the CADEC corpus [e.g. 13 , 38 – 40 ] and post-/sentence-level AE classification for the TwiMed corpus [e.g. 41 ], with the mapping of the event mention to a terminology being ignored.…”

Section: Introductionmentioning

confidence: 99%

Prospective Evaluation of Adverse Event Recognition Systems in Twitter: Results from the Web-RADR Project

et al. 2020

View full text Add to dashboard Cite

Introduction A large number of studies on systems to detect and sometimes normalize adverse events (AEs) in social media have been published, but evidence of their practical utility is scarce. This raises the question of the transferability of such systems to new settings. Objectives The aims of this study were to develop an AE recognition system, prospectively evaluate its performance on an external benchmark dataset and identify potential factors influencing the transferability of AE recognition systems. Methods A pipeline based on dictionary lookups and logistic regression classifiers was developed using a proprietary dataset of 196,533 Tweets manually annotated for AE relations and prospectively evaluated the system on the publicly available WEB-RADR reference dataset, exploring different aspects affecting transferability. Results Our system achieved 0.53 precision, 0.52 recall and 0.52 F1-score on the development test set; however, when applied to the WEB-RADR reference dataset, system performance dropped to 0.38 precision, 0.20 recall and 0.26 F1-score. Similarly, a previously published method aiming at automatically detecting adverse event posts reported 0.5 precision, 0.92 recall and 0.65 F1-score on thus another dataset, while performance on the WEB-RADR reference dataset was reduced to 0.37 precision, 0.63 recall and 0.46 F1-score. We identified four potential factors leading to poor transferability: overfitting, selection bias, label bias and prevalence. Conclusion We warn the community about a potentially large discrepancy between the expected performance of automated AE recognition systems based on published results and the actual observed performance on independent data. This study highlights the difficulty of implementing an all-purpose system for automatic adverse event recognition in Twitter, which could explain the lack of such systems in practical pharmacovigilance settings. Our recommendation is to use benchmark independent datasets, such as the WEB-RADR reference, to investigate the transferability of the adverse event recognition systems and ultimately enforce rigorous comparisons across studies on the task.

show abstract

Section: Introductionmentioning

confidence: 99%

Prospective Evaluation of Adverse Event Recognition Systems in Twitter: Results from the Web-RADR Project

et al. 2020

View full text Add to dashboard Cite

show abstract

“…We only use ADE annotations because only the ADEs involve discontinuous annotations. This also allows us to compare our results directly against previously reported results (Metke-Jimenez and Karimi, 2016;Tang et al, 2018). ShARe 13 and 14 focus on the identification of disorder mentions in clinical notes, including discharge summaries, electrocardiogram, echocardiogram, and radiology reports (Johnson et al, 2016 mapped to a concept in the disorder semantic group of SNOMED-CT (Cornet and de Keizer, 2008).…”

Section: Data Setsmentioning

confidence: 82%

An Effective Transition-based Model for Discontinuous NER

Dai¹,

Karimi²,

Hachey³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Unlike widely used Named Entity Recognition (NER) data sets in generic domains, biomedical NER data sets often contain mentions consisting of discontinuous spans. Conventional sequence tagging techniques encode Markov assumptions that are efficient but preclude recovery of these mentions. We propose a simple, effective transition-based model with generic neural encoding for discontinuous NER. Through extensive experiments on three biomedical data sets, we show that our model can effectively recognize discontinuous mentions without sacrificing the accuracy on continuous mentions.

show abstract

“…In future, one may investigate neural networks [21][22][23][24][25][26] based methodologies for better planning, prediction, and policies for not only KPK region but also other similar regions all over the world.…”

Section: Discussionmentioning

confidence: 99%

A Novel Study to Predict Trends and Policies for Mobile Communication in Multienvironment Regions

Khattak

Khan

2018

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

The research community has growing interest in the field of mobile phone applications due to its extensive usage in voice, data, video, and their combinations including data-voice and voice-video. The trends of mobile usage have paramount significance to formulate effective policies, to analyze human behavior, reliable prediction, and planning, for development and prosperity of any region in the globe. In this research, we explore the usage of mobile communication to analyze these trends; for this, we select Khyber Pakhtunkhwa (KPK), Pakistan, a multienvironment based on farmers, students, employees, and businessmen. A field survey is carried out by designing a detailed structural questionnaire for viable collection of data to have superior analysis of different multienvironment based classes. A statistical analysis is performed through hypotheses and chi-square test on a large dataset based on sufficient number of observations collected through primary survey for each class. The survey results are provided in number of graphical and numerical illustrations to predict the mobile usage trends, evidently and effectively, of the people of KPK. This study has practical and significant implications in understanding the behavioral intensions of the individuals making policies about mobile phone usage.

show abstract

Recognizing Continuous and Discontinuous Adverse Drug Reaction Mentions from Social Media Using LSTM‐CRF

Cited by 37 publications

References 21 publications

Prospective Evaluation of Adverse Event Recognition Systems in Twitter: Results from the Web-RADR Project

Prospective Evaluation of Adverse Event Recognition Systems in Twitter: Results from the Web-RADR Project

An Effective Transition-based Model for Discontinuous NER

A Novel Study to Predict Trends and Policies for Mobile Communication in Multienvironment Regions

Contact Info

Product

Resources

About