Validation and Topic-driven Ranking for Biomedical Hypothesis Generation Systems

Abstract

Literature underpins research, providing the foundation for new ideas. But as the pace of science accelerates, many researchers struggle to stay current. To expedite their searches, some scientists leverage hypothesis generation (HG) systems, which can automatically inspect published papers to uncover novel implicit connections. With no foreseeable end to the driving pace of research, we expect these systems will become crucial for productive scientists, and later form the basis of intelligent automated discovery systems. Yet, many resort to expert analysis to validate such systems. This process is slow, hard to reproduce, and takes time away from other researchers. Therefore, we present a novel method to validate HG systems, which both scales to large validation sets and does not require expert input. We also introduce a number of new metrics to automatically identify plausible generated hypotheses. Through the study of published, highly cited, and noise predicates, we devise a validation challenge, which allows us to evaluate the performance of a HG system. Using an in-progress system, MOLIERE, as a case-study, we show the utility of our validation and ranking methods. So that others may reproduce our results, we provide our code, validation data, and results at bit.ly/2EtVshN.

Data-driven hypothesis generation among inexperienced clinical researchers: A comparison of secondary data analyses with visualization (VIADS) and other tools

“…Many researchers believe that new analytical tools offer opportunities to reveal further insights and new patterns in existing data to facilitate hypothesis generation [1,28,41,42,50]. We developed the underlying algorithms (determine what VIADS can do) [38,39] and the publicly accessible online tool-VIADS [37,51,52] to provide new ways of summarizing, comparing, and visualizing datasets.…”

“…Many of these efforts were based on Swanson's ABC Model [24][25][26]. Several research teams explored automatic literature systems for generating [28,29] and validating [30] or enriching hypotheses [31]. However, the studies recognized the complexity of the hypothesis generation process and concluded that it does not seem feasible to generate hypotheses completely automatically [19][20][21]24,32].…”

T-PAIR: Temporal Node-pair Embedding for Automatic Biomedical Hypothesis Generation

“…They applied random walk to capture the structural information of the scholarly graph. Some other methods [2], [18], [19] proposed the incorporation of machine learning techniques such as Latent Dirichlet Allocation (LDA), clustering, and topical phrase mining. The previously mentioned methods all focused mostly on static scholarly graphs.…”

“…The foundation of both expertise and ideas is rooted in literature, as literature provides the background for new knowledge and information. Thus, new hypotheses with minimum uncertainty about undiscovered knowledge can be made from already published scholarly literature [2], [3]. In this paper, we focus on new hypotheses about whether two scientific terms/concepts are relevant to each other, given that there is no direct link between them at the current knowledge scope.…”

“…The second group covers methods that use a combination of advanced machine learning strategies to extract and analyze hidden connections from scientific publications. These methods include but not limited to association rules [10], [12], [16], text mining [13], [17], clustering and topic modeling [2], [18], [19], and others [14], [20], [21]. However, most of the previous studies fail to capture and utilize the dynamic evolution of the entity meaning, which can provide crucial information on inferring the future connectivity of the entities (e.g., medical terms).…”