Extracting Conceptual Knowledge from Natural Language Text Using Maximum Likelihood Principle

Abstract

Domain-specific knowledge graphs constructed from natural language text are ubiquitous in today's world. In many such scenarios the base text, from which the knowledge graph is constructed, concerns itself with practical, on-hand, actual or ground-reality information about the domain. Product documentation in software engineering domain are one example of such base texts. Other examples include blogs and texts related to digital artifacts, reports on emerging markets and business models, patient medical records, etc. Though the above sources contain a wealth of knowledge about their respective domains, the conceptual knowledge on which they are based is often missing or unclear. Access to this conceptual knowledge can enormously increase the utility of available data and assist in several tasks such as knowledge graph completion, grounding, querying, etc. Additionally, this conceptual knowledge can also be used to find common semantics across different knowledge graphs. Our contributions in this paper are twofold. First, we propose a novel Markovian stochastic model for document generation from conceptual knowledge. This model is based on the assumption that a writer generates successive sentences of a text only based on her current context as well as the domain-specific conceptual knowledge in her mind. The uniqueness of our approach lies in the fact that the conceptual knowledge in the writer's mind forms a component of the parameter set λ of our stochastic model. Secondly, we solve the inverse problem of learning the best conceptual knowledge from a given document O, by finding model parameters λ * which maximize the likelihood of generating document O over all possible parameter values. This likelihood maximization is done using an application of Baum-Welch algorithm, which is a known special case of Expectation-Maximization (EM) algorithm. We run our conceptualization algorithm on several well-known natural language sources and obtain very encouraging results. The results of our extensive experiments concur with the hypothesis that the information contained in these sources has a well-defined and rigorous underlying conceptual structure, which can be discovered using our method.