Probabilistic Databases with an Infinite Open-World Assumption

Abstract

Probabilistic databases (PDBs) introduce uncertainty into relational databases by specifying probabilities for several possible instances. Traditionally, they are finite probability spaces over database instances. Such finite PDBs inherently make a closed-world assumption: non-occurring facts are assumed to be impossible, rather than just unlikely. As convincingly argued by Ceylan et al. (KR '16), this results in implausibilities and clashes with intuition. An open-world assumption, where facts not explicitly listed may have a small positive probability can yield more reasonable results. The corresponding open-world model of Ceylan et al., however, assumes that all entities in the PDB come from a fixed finite universe.In this work, we take one further step and propose a model of "truly" open-world PDBs with an infinite universe. This is natural when we consider entities from typical domains such as integers, real numbers, or strings. While the probability space might become infinitely large, all instances of a PDB remain finite. We provide a sound mathematical framework for infinite PDBs generalizing the existing theory of finite PDBs. Our main results are concerned with countable, tuple-independent PDBs; we present a generic construction showing that such PDBs exist in the infinite and provide a characterization of their existence. This construction can be used to give an open-world semantics to finite PDBs. The construction can also be extended to so-called block-independent-disjoint probabilistic databases.Algorithmic questions are not the focus of this paper, but we show how query evaluation algorithms can be lifted from finite PDBs to perform approximate evaluation (with an arbitrarily small additive approximation error) in countably infinite tuple-independent PDBs.

Generative Datalog with Continuous Distributions

“…Unfortunately, generalizing from discrete to continuous distributions usually comes with substantial mathematical overhead. While several systems [2,24,35] handle continuous probability distributions, only recently [21,22], Grohe and Lindner proposed a general framework for rigorously dealing with probabilistic databases over continuous domains. Moreover, they establish basic properties such as the measurability of relational calculus and Datalog queries, which in turn allows for formally specifying the semantics of queries over continuous probabilistic databases.…”

On Constrained Open-World Probabilistic Databases

“…Problems outside of simple query evaluation are also points of interest for PDBs, for example the most probable database [Gribkoff et al, 2014b], or of ranking the top-k results [Ré et al, 2007]. In the context of OpenPDBs in particular, Grohe and Lindner [2018] study the notion of an infinite open world, using techniques from analysis to explore when this is feasible.…”

Open-world probabilistic databases: Semantics, algorithms, complexity

“…OpenPDBs also motivated further research to extend the open-world probabilistic database model to have schema-level constraints on completion probabilities [78]. OpenPDBs are defined over a finite domain, and the work of Grohe and Lindner [79] extends the open-world probabilistic database model to infinite universes.…”