Supervised Relation Extraction Between Suicide-Related Entities and Drugs: Development and Usability Study of an Annotated PubMed Corpus (Preprint)

Abstract

BACKGROUND Drug-induced suicide has been debated as a crucial issue in clinical and public health research. Published research articles are valuable data resources to find information on drugs associated with suicidal adverse events. It is essential to apply an automated process to extract such information and rapidly detect drugs related to suicide risk. Still, such a process is not well established, and there has also been little dataset to train and validate models to classify drug-induced suicide. OBJECTIVE This study aims to build a drug-suicide relations (DSR) corpus of annotation of entities for drugs and suicidal events and their relations. To confirm the effectiveness of the DSR corpus, we evaluate the performance of a relation classification model in conjunction with various embeddings applying the corpus. METHODS We collect abstracts and titles of research articles associated with drugs and suicide from PubMed. We conduct manual annotation for entities of drug and suicide and their relation (as adverse drug events, treatment, suicide means, and miscellaneous) at the sentence level. To reduce the manual annotation effort, we run the annotation procedure after selecting sentences with pre-trained zero-shot classifier or selecting only sentences with both drug and suicide keywords. We train a relation classification model using various BERT (Bidirectional Encoder Representations from Transformers) embeddings with the proposed corpus. We then evaluate the performance of the model to determine the best BERT-based embedding that best suits our corpus. RESULTS Our corpus comprises 11,894 sentences from titles and abstracts of research articles extracted from PubMed. Each sentence is annotated with (1) drug and suicide entities and (2) the relation between these two entities, including adverse drug events, treatment, means, miscellaneous, and others. All relation classification models that we observe are fine-tuned based on our corpus. The models achieve F1 scores above 0.8 in detecting sentences of suicidal adverse events, regardless of the pre-trained model and dataset properties. CONCLUSIONS To the best of our knowledge, the proposed corpus is the first and the most extensive corpus targeting drug-suicide relations.