Quantifying and filtering knowledge generated by literature based discovery

Abstract

BackgroundLiterature based discovery (LBD) automatically infers missed connections between concepts in literature. It is often assumed that LBD generates more information than can be reasonably examined.MethodsWe present a detailed analysis of the quantity of hidden knowledge produced by an LBD system and the effect of various filtering approaches upon this. The investigation of filtering combined with single or multi-step linking term chains is carried out on all articles in PubMed.ResultsThe evaluation is carried out using both replication of existing discoveries, which provides justification for multi-step linking chain knowledge in specific cases, and using timeslicing, which gives a large scale measure of performance.ConclusionsWhile the quantity of hidden knowledge generated by LBD can be vast, we demonstrate that (a) intelligent filtering can greatly reduce the number of hidden knowledge pairs generated, (b) for a specific term, the number of single step connections can be manageable, and (c) in the absence of single step hidden links, considering multiple steps can provide valid links.

A systematic review on literature-based discovery workflow

“…The Data unit of analysis denotes the types of data extracted from the above-discussed data types to represent the knowledge associations. Since most of the LBD research performed in medicine, the most common term representation is using UMLS and MeSH (Lever et al, 2017;Preiss & Stevenson, 2017). Apart from these two medical resources, other medical databases such as Entrez Gene , HUGO (Petric et al, 2014), LocusLink (Hristovski et al, 2005), OMIM (Hristovski et al, 2003) and PharmGKB (Kim & Park, 2016) have also being used extract data units.…”

“…For example, terms such as ''drug'', ''treatment'' can be considered as general terms in the medical domain. Using stop words to remove uninformative terms is a popular filtering technique used (Lever et al, 2017;Preiss & Stevenson, 2017;Sebastian, Siew & Orimaye, 2017b). Stop word list could be either manually created, obtained from other resources, or automatically generated.…”

“…Stop word list could be either manually created, obtained from other resources, or automatically generated. (1) Manually created: A popular example of this category is the stop word list created for the Arrowsmith project (Smalheiser, 2005) that have nearly 9500 terms by 2006 (Preiss & Stevenson, 2017). However, manual development of a stop words is costly, and time-consuming.…”

“…(3) Automatically generated: Some studies (Preiss & Stevenson, 2017;Hu et al, 2010) have automatically created their stop word lists by employing different techniques. The most common way is eliminating terms that appear above a user-defined threshold (Pratt & Yetisgen-Yildiz, 2003).…”

“…Expert/user-based filtering: Expert/user-based filtering (Gubiani et al, 2017;Preiss & Stevenson, 2017) involves the decision of an expert/user to remove uninteresting associations. For example, most of the semantic category filter requires user-defined semantic types/groups to perform the filtering.…”