Jointly Embedding Entities and Text with Distant Supervision

Newman-Griffis, Denis; Lai, Albert M.; Fosler‐Lussier, Eric

doi:10.18653/v1/w18-3026

Cited by 30 publications

(32 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, there is significant research into strategies for learning neural representations of entities in knowledge bases and coding systems. Past work has investigated diverse approaches, such as leveraging rich semantic information from knowledge base structure and web-scale annotated corpora (34,97,98), utilizing definitions of word senses (similar to our use of ICF definitions) (99,100), and combining terminologies with targeted selection of training corpora to learn applicationtailored concept representations (101,102). While most of the research on entity representations requires resources not yet available for FSI (e.g., large, annotated corpora; well-developed terminologies; robust and interconnected knowledge graph structure), all present significant opportunities to advance FSI coding technologies as more resources are developed.…”

Section: Alternative Coding Approachesmentioning

confidence: 99%

Automated Coding of Under-Studied Medical Concept Domains: Linking Physical Activity Reports to the International Classification of Functioning, Disability, and Health

Newman-Griffis¹,

Fosler‐Lussier²

2021

Front. Digit. Health

Self Cite

View full text Add to dashboard Cite

Linking clinical narratives to standardized vocabularies and coding systems is a key component of unlocking the information in medical text for analysis. However, many domains of medical concepts, such as functional outcomes and social determinants of health, lack well-developed terminologies that can support effective coding of medical text. We present a framework for developing natural language processing (NLP) technologies for automated coding of medical information in under-studied domains, and demonstrate its applicability through a case study on physical mobility function. Mobility function is a component of many health measures, from post-acute care and surgical outcomes to chronic frailty and disability, and is represented as one domain of human activity in the International Classification of Functioning, Disability, and Health (ICF). However, mobility and other types of functional activity remain under-studied in the medical informatics literature, and neither the ICF nor commonly-used medical terminologies capture functional status terminology in practice. We investigated two data-driven paradigms, classification and candidate selection, to link narrative observations of mobility status to standardized ICF codes, using a dataset of clinical narratives from physical therapy encounters. Recent advances in language modeling and word embedding were used as features for established machine learning models and a novel deep learning approach, achieving a macro-averaged F-1 score of 84% on linking mobility activity reports to ICF codes. Both classification and candidate selection approaches present distinct strengths for automated coding in under-studied domains, and we highlight that the combination of (i) a small annotated data set; (ii) expert definitions of codes of interest; and (iii) a representative text corpus is sufficient to produce high-performing automated coding systems. This research has implications for continued development of language technologies to analyze functional status information, and the ongoing growth of NLP tools for a variety of specialized applications in clinical care and research.

show abstract

Section: Alternative Coding Approachesmentioning

confidence: 99%

Automated Coding of Under-Studied Medical Concept Domains: Linking Physical Activity Reports to the International Classification of Functioning, Disability, and Health

Newman-Griffis¹,

Fosler‐Lussier²

2021

Front. Digit. Health

Self Cite

View full text Add to dashboard Cite

show abstract

“…Several studies [37,39,46] proposed a hybrid between entity and word embeddings by employing a loss function, which includes both a TransE-based component to model relations between entities and a word2vec-based component to model semantic relations between the words along with the third component, whose purpose is to align entity and word embeddings obtained by the first two components. In [23] authors take a different approach by learning word and entity embeddings without utilizing relations between entities from a knowledge graph and instead relying only on an unannotated corpus of text. None of the previously proposed approaches for learning joint word and entity embedding spaces were proposed specifically for entity search in a knowledge graph, and thus ignore important information, such as knowledge graph structural components.…”

Section: Related Workmentioning

confidence: 99%

Joint Word and Entity Embeddings for Entity Retrieval from a Knowledge Graph

Nikolaev

Kotov

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Recent years have witnessed the emergence of novel models for ad-hoc entity search in knowledge graphs of varying complexity. Since these models are based on direct term matching, their accuracy can suffer from a mismatch between vocabularies used in queries and entity descriptions. Although successful applications of word embeddings and knowledge graph entity embeddings to address the issues of vocabulary mismatch in ad-hoc document retrieval and knowledge graph noisiness and incompleteness, respectively, have been reported in recent literature, the utility of joint word and entity embeddings for entity search in knowledge graphs has been relatively unexplored. In this paper, we propose Knowledge graph Entity and Word Embedding for Retrieval (KEWER), a novel method to embed entities and words into the same low-dimensional vector space, which takes into account a knowledge graph's local structure and structural components, such as entities, attributes, and categories, and is designed specifically for entity search. KEWER is based on random walks over the knowledge graph and can be considered as a hybrid of word and network embedding methods. Similar to word embedding methods, KEWER utilizes contextual co-occurrences as training data, however, it treats words and entities as different objects. Similar to network embedding methods, KEWER takes into account knowledge graph's local structure, however, it also differentiates between structural components. Experiments on publicly available entity search benchmarks and state-ofthe-art word and joint word and entity embedding methods indicate that a combination of KEWER and BM25F results in a consistent improvement in retrieval accuracy over BM25F alone.

show abstract

“…SPECTER paper embeddings have been shown to successfully capture paper similarity [ 18 ] and are available for all papers in CORD-19. Also available for papers in CORD-19 are clinical concept embeddings trained using the JET algorithm [ 60 ], relation embeddings trained using SeVeN [ 28 ] and network co-occurrence embeddings [ 63 ] for biomedical entities computed using CORD-19-on-FHIR. Embeddings capture text similarity and can be used to retrieve similar texts, e.g.…”

Section: Text Mining Modeling Resourcesmentioning

confidence: 99%

Text mining approaches for dealing with the rapidly expanding literature on COVID-19

Wang

2020

Briefings in Bioinformatics

View full text Add to dashboard Cite

More than 50 000 papers have been published about COVID-19 since the beginning of 2020 and several hundred new papers continue to be published every day. This incredible rate of scientific productivity leads to information overload, making it difficult for researchers, clinicians and public health officials to keep up with the latest findings. Automated text mining techniques for searching, reading and summarizing papers are helpful for addressing information overload. In this review, we describe the many resources that have been introduced to support text mining applications over the COVID-19 literature; specifically, we discuss the corpora, modeling resources, systems and shared tasks that have been introduced for COVID-19. We compile a list of 39 systems that provide functionality such as search, discovery, visualization and summarization over the COVID-19 literature. For each system, we provide a qualitative description and assessment of the system’s performance, unique data or user interface features and modeling decisions. Many systems focus on search and discovery, though several systems provide novel features, such as the ability to summarize findings over multiple documents or linking between scientific articles and clinical trials. We also describe the public corpora, models and shared tasks that have been introduced to help reduce repeated effort among community members; some of these resources (especially shared tasks) can provide a basis for comparing the performance of different systems. Finally, we summarize promising results and open challenges for text mining the COVID-19 literature.

show abstract

Jointly Embedding Entities and Text with Distant Supervision

Cited by 30 publications

References 27 publications

Automated Coding of Under-Studied Medical Concept Domains: Linking Physical Activity Reports to the International Classification of Functioning, Disability, and Health

Automated Coding of Under-Studied Medical Concept Domains: Linking Physical Activity Reports to the International Classification of Functioning, Disability, and Health

Joint Word and Entity Embeddings for Entity Retrieval from a Knowledge Graph

Text mining approaches for dealing with the rapidly expanding literature on COVID-19

Contact Info

Product

Resources

About