Triplet-Trained Vector Space and Sieve-Based Search Improve Biomedical Concept Normalization

Abstract

Concept normalization, the task of linking textual mentions of concepts to concepts in an ontology, is critical for mining and analyzing biomedical texts. We propose a vector-space model for concept normalization, where mentions and concepts are encoded via transformer networks that are trained via a triplet objective with online hard triplet mining. The transformer networks refine existing pre-trained models, and the online triplet mining makes training efficient even with hundreds of thousands of concepts by sampling training triples within each mini-batch. We introduce a variety of strategies for searching with the trained vector-space model, including approaches that incorporate domain-specific synonyms at search time with no model retraining. Across five datasets, our models that are trained only once on their corresponding ontologies are within 3 points of state-of-the-art models that are retrained for each new domain. Our models can also be trained for each domain, achieving new state-of-the-art on multiple datasets.

Chemical identification and indexing in PubMed full-text articles using deep learning and heuristics

“…The authors used a BiLSTM-CRF tagger for NER and adopted a graph-based model to rank concept candidates for each entity mention. Many other recent works have also tackled hybrid approaches ( 59 ) and edit patterns ( 60 ), analyzed the problem of ambiguity ( 61 ), explored transformer networks trained via a triplet objective ( 62 ) and multi-task frameworks ( 63 ) and experimented using large-scale datasets ( 64 ).…”