Algorithmic statistics

Gács, Péter; Tromp, John; Vitányi, Paul M. B.

doi:10.1109/18.945257

Cited by 100 publications

(159 citation statements)

References 21 publications

Supporting

Mentioning

154

Contrasting

Unclassified

Order By: Relevance

“…Then the extended common cause principle (Theorem 2) relates stochastic dependence to a property of all Bayesian networks that include the observations. However, the result holds for more general observations (such as binary strings) and for more general notions of mutual information (such as algorithmic mutual information [8]). Therefore, we introduce an "axiomatized" version of mutual information in the following section and describe how it can be connected to a DAG.…”

Section: Introductionmentioning

confidence: 99%

Information-Theoretic Inference of Common Ancestors

Steudel

2015

Entropy

View full text Add to dashboard Cite

A directed acyclic graph (DAG) partially represents the conditional independence structure among observations of a system if the local Markov condition holds, that is if every variable is independent of its non-descendants given its parents. In general, there is a whole class of DAGs that represents a given set of conditional independence relations. We are interested in properties of this class that can be derived from observations of a subsystem only. To this end, we prove an information-theoretic inequality that allows for the inference of common ancestors of observed parts in any DAG representing some unknown larger system. More explicitly, we show that a large amount of dependence in terms of mutual information among the observations implies the existence of a common ancestor that distributes this information. Within the causal interpretation of DAGs, our result can be seen as a quantitative extension of Reichenbach's principle of common cause to more than two variables. Our conclusions are valid also for non-probabilistic observations, such as binary strings, since we state the proof for an axiomatized notion of "mutual information" that includes the stochastic as well as the algorithmic version.

show abstract

Section: Introductionmentioning

confidence: 99%

Information-Theoretic Inference of Common Ancestors

Steudel

2015

Entropy

View full text Add to dashboard Cite

show abstract

“…Let be a set of minimal Kolmogorov complexity among the sets with and . Then Lemma 3: For every set with (6) up to a additive term. Proof: Inequality (6) means that that is, .…”

Section: Definitionmentioning

confidence: 99%

“…This function and the associated Kolmogorov sufficient statistic are partially treated in [19], [24], and [6] and analyzed in detail in [22]. We will show that the structure function approach can be generalized to give an approach to rate distortion and denoising of individual data.…”

Section: A Related Workmentioning

confidence: 99%

Rate Distortion and Denoising of Individual Data Using Kolmogorov Complexity

Vereshchagin

Vitányi

2010

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

Abstract-We examine the structure of families of distortion balls from the perspective of Kolmogorov complexity. Special attention is paid to the canonical rate-distortion function of a source word which returns the minimal Kolmogorov complexity of all distortion balls containing that word subject to a bound on their cardinality. This canonical rate-distortion function is related to the more standard algorithmic rate-distortion function for the given distortion measure. Examples are given of list distortion, Hamming distortion, and Euclidean distortion. The algorithmic rate-distortion function can behave differently from Shannon's rate-distortion function. To this end, we show that the canonical rate-distortion function can and does assume a wide class of shapes (unlike Shannon's); we relate low algorithmic mutual information to low Kolmogorov complexity (and consequently suggest that certain aspects of the mutual information formulation of Shannon's rate-distortion function behave differently than would an analogous formulation using algorithmic mutual information); we explore the notion that low Kolmogorov complexity distortion balls containing a given word capture the interesting properties of that word (which is hard to formalize in Shannon's theory) and this suggests an approach to denoising.

show abstract

“…They were studied in [3,15]. To define non-stochasticity rigorously we have to introduce the notion of the profile of x, which represents the parameters of possible explanations for x.…”

Section: Algorithmic Statisticsmentioning

confidence: 99%

Some Properties of Antistochastic Strings

Milovanov

2015

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Algorithmic statistics is a part of algorithmic information theory (Kolmogorov complexity theory) that studies the following task: given a finite object x (say, a binary string), find an 'explanation' for it, i.e., a simple finite set that contains x and where x is a 'typical element'. Both notions ('simple' and 'typical') are defined in terms of Kolmogorov complexity.It was found that this cannot be achieved for some objects: there are some "non-stochastic" objects that do not have good explanations. In this paper we study the properties of maximally non-stochastic objects; we call them "antistochastic".It turns out the antistochastic strings have the following property (Theorem 6): if an antistochastic string has complexity k, then any k bit of information about x are enough to reconstruct x (with logarithmic advice). In particular, if we erase all bits of this antistochastic string except for k, the erased bits can be restored from the remaining ones (with logarithmic advice). As a corollary we get the existence of good list-decoding codes with erasures (or other ways of deleting part of the information).Antistochastic strings can also be used as a source of counterexamples in algorithmic information theory. We show that the symmetry of information property fails for total conditional complexity for antistochastic strings.

show abstract

Algorithmic statistics

Cited by 100 publications

References 21 publications

Information-Theoretic Inference of Common Ancestors

Information-Theoretic Inference of Common Ancestors

Rate Distortion and Denoising of Individual Data Using Kolmogorov Complexity

Some Properties of Antistochastic Strings

Contact Info

Product

Resources

About