Detecting Personal Medication Intake in Twitter: An Annotated
            Corpus and Baseline Classification System

Klein, Ari Z.; Sarker, Abeed; Rouhizadeh, Masoud; O’Connor, Karen; González, Graciela

doi:10.18653/v1/w17-2316

Cited by 34 publications

(37 citation statements)

References 19 publications

(23 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We collected all the data from Twitter via the public streaming API, using generic and trade names for medications, along with their common misspellings, totaling over 250 keywords. For subtasks-1 and -2, the annotated datasets for training were made available to the public with our prior publications, 21 , 27 , 28 while subtask-3 included previously unpublished data. Evaluation data were not made public at the time of the workshop.…”

Section: Methodsmentioning

confidence: 99%

Data and systems for medication-related text classification and concept normalization from Twitter: insights from the Social Media Mining for Health (SMM4H)-2017 shared task

Sarker

Belousov

Friedrichs³

et al. 2018

Journal of the American Medical Informatics Association

Self Cite

View full text Add to dashboard Cite

ObjectiveWe executed the Social Media Mining for Health (SMM4H) 2017 shared tasks to enable the community-driven development and large-scale evaluation of automatic text processing methods for the classification and normalization of health-related text from social media. An additional objective was to publicly release manually annotated data.Materials and MethodsWe organized 3 independent subtasks: automatic classification of self-reports of 1) adverse drug reactions (ADRs) and 2) medication consumption, from medication-mentioning tweets, and 3) normalization of ADR expressions. Training data consisted of 15 717 annotated tweets for (1), 10 260 for (2), and 6650 ADR phrases and identifiers for (3); and exhibited typical properties of social-media-based health-related texts. Systems were evaluated using 9961, 7513, and 2500 instances for the 3 subtasks, respectively. We evaluated performances of classes of methods and ensembles of system combinations following the shared tasks.ResultsAmong 55 system runs, the best system scores for the 3 subtasks were 0.435 (ADR class F1-score) for subtask-1, 0.693 (micro-averaged F1-score over two classes) for subtask-2, and 88.5% (accuracy) for subtask-3. Ensembles of system combinations obtained best scores of 0.476, 0.702, and 88.7%, outperforming individual systems.DiscussionAmong individual systems, support vector machines and convolutional neural networks showed high performance. Performance gains achieved by ensembles of system combinations suggest that such strategies may be suitable for operational systems relying on difficult text classification tasks (eg, subtask-1).ConclusionsData imbalance and lack of context remain challenges for natural language processing of social media text. Annotated data from the shared task have been made available as reference standards for future studies (http://dx.doi.org/10.17632/rxwfb3tysd.1).

show abstract

Section: Methodsmentioning

confidence: 99%

Data and systems for medication-related text classification and concept normalization from Twitter: insights from the Social Media Mining for Health (SMM4H)-2017 shared task

Sarker

Belousov

Friedrichs³

et al. 2018

Journal of the American Medical Informatics Association

Self Cite

View full text Add to dashboard Cite

show abstract

“…Since there are no trainable datasets that we could make use of, we created a dataset utilizing annotated datasets from different sources [18][19][20][21] . We emphasize that we did not annotate or create any annotated set of tweets ourselves.…”

Section: Classificationmentioning

confidence: 99%

“…We collected 259,042 tweets that only have drug strings from multiple papers on pharmacovigilance using social media [18][19][20][21] and downloaded all the tweets available through them. These tweets were annotated by different annotators as part of their research.…”

Section: Classical Modelsmentioning

confidence: 99%

Mining Archive.org’s Twitter Stream Grab for Pharmacovigilance Research Gold

Tekumalla

Asl

Banda

2019

Preprint

View full text Add to dashboard Cite

In the last few years Twitter has become an important resource for the identification of Adverse Drug Reactions (ADRs), monitoring flu trends, and other pharmacovigilance and general research applications. Most researchers spend their time crawling Twitter, buying expensive pre-mined datasets, or tediously and slowly building datasets using the limited Twitter API. However, there are a large number of datasets that are publicly available to researchers which are underutilized or unused. In this work, we demonstrate how we mined over 9.4 billion Tweets from archive.org's Twitter stream grab using a drug-term dictionary and plenty of computing power. Knowing that not everything that shines is gold, we used pre-existing drug-related datasets to build machine learning models to filter our findings for relevance. In this work we present our methodology and the 3,346,758 identified tweets for public use in future research.

show abstract

“…In the health sector, twitter was used, for example, to monitor and predict the spread of influenza [3]- [5]. It was also used to monitor the adverse effect of medications in [6] and [7], track medication adherence in [8] and for the understanding of the well-being of military populations in [9].…”

Section: Related Workmentioning

confidence: 99%

“…Detecting and subsequently eliminating the irrelevant tweets can be achieved by using a classifier. Several classifiers were used in [8] for the study of medication adherence using Twitter. These classifiers include Bayesian networks, random forests, logistic regression, and support vector machines (SVM).…”

Section: Related Workmentioning

confidence: 99%