Linear List-Approximation for Short Programs (or the Power of a Few Random Bits)

Bauwens, Bruno; Zimand, Marius

doi:10.1109/ccc.2014.32

Cited by 15 publications

(34 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kobayashi [20] proposed the following uniform notion of compressibility for streams. 8 Definition 1.8 (Kobayashi [20]). Given a function f : N → N, we say X is f -compressible if there exists Y which computes X with oracle-use bounded above by f .…”

Section: The Online Compression Of Binary Streams and Kolmogorov Compmentioning

confidence: 99%

See 1 more Smart Citation

Compression of Data Streams Down to Their Information Content

Barmpalias

Lewis-Pye

2019

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

According to Kolmogorov complexity, every finite binary string is compressible to a shortest code -its information content -from which it is effectively recoverable. We investigate the extent to which this holds for infinite binary sequences (streams). We devise a new coding method which uniformly codes every stream X into an algorithmically random stream Y, in such a way that the first n bits of X are recoverable from the first I(X ↾ n ) bits of Y, where I is any partial computable information content measure which is defined on all prefixes of X, and where X ↾ n is the initial segment of X of length n. As a consequence, if g is any computable upper bound on the initial segment prefix-free complexity of X, then X is computable from an algorithmically random Y with oracle-use at most g. Alternatively (making no use of such a computable bound g) one can achieve an oracle-use bounded above by K(X ↾ n ) + log n. This provides a strong analogue of Shannon's source coding theorem for algorithmic information theory.A fruitful way to quantify the complexity of a finite object such as a string σ in a finite alphabet 1 is to consider the length of the shortest binary program which prints σ. This fundamental idea gives rise to a theory of algorithmic information and compression, which is based on the theory of computation and was pioneered by Kolmogorov [21] and Solomonoff [41]. The Kolmogorov complexity of a binary string σ is the length of the shortest program that outputs σ with respect to a fixed universal Turing machine. The use of prefix-free machines in the definition of Kolmogorov complexity was pioneered by Levin [26] and Chaitin [10], and allowed for the development of a robust theory of incompressibility and algorithmic randomness for streams (i.e. infinite binary sequences). Information content measures, defined by Chaitin [11] after Levin [26], are functions that assign a positive integer value to each binary string, representing the amount of information contained in the string.Definition 1.1 (Information content measure). A partial function I from strings to N is an information content measure if it is right-c.e. 2 and I(σ)↓ 2 −I(σ) is finite.Prefix-free Kolmogorov complexity can be characterized as the minimum (modulo an additive constant) information content measure. If K(σ) denotes the prefix-free Kolmogorov complexity of the string σ, then for c ∈ N we say that σ is c-incompressible if K(σ) ≥ |σ| − c. It is a basic fact concerning Kolmogorov complexity that for some universal constant c: every string σ has a shortest code σ * which is itself c-incompressible.(1)Our goal is to investigate the extent to which the above fact holds in an infinite setting, i.e. for streams instead of strings. In the context of Kolmogorov complexity, algorithmic randomness is defined as incompressibility. 3 So (1) can be read as follows: we can uniformly code each string σ into an algorithmically random string of length K(σ). In order to formalise an infinitary analogue of this statement, we need to make use of oracle-machine ...

show abstract

Section: The Online Compression Of Binary Streams and Kolmogorov Compmentioning

confidence: 99%

“…Note that there exists a 1 in the trace of ℓ i , i < t on a position which is at most ℓ t , otherwise the weight of ℓ i , i < t + 1 would exceed 1. Let p be the largest such position and consider the string µ ∈ F t which corresponds to this position according to (8). Then:…”

Section: Plain Kraft-chaitin Requests and The Greedy Solutionmentioning

confidence: 99%

Compression of Data Streams Down to Their Information Content

Barmpalias

Lewis-Pye

2019

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…Their result can be reformulated as follows. 2 Theorem 1.1 ( [BZ14]). There exist a probabilistic algorithm E and a deterministic algorithm D such that E runs in polynomial-time, and for all n-bit strings x and y and for every rational number δ > 0, 1.…”

Section: Introductionmentioning

confidence: 99%

“…The gain is that this decoding procedure halts even with incorrect help bits, even though the result may not be the desired x. Next, Bauwens and Zimand [BZ14] have eliminated the help bits in Muchnik's theorem, at the cost of introducing a small error probability. Their result can be reformulated as follows.…”

Section: Introductionmentioning

confidence: 99%

Kolmogorov complexity version of Slepian-Wolf coding

Zimand

2017

Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing

Self Cite

View full text Add to dashboard Cite

Alice and Bob are given two correlated n-bit strings x 1 and, respectively, x 2 , which they want to losslessly compress and send to Zack. They can either collaborate by sharing their strings, or work separately. We show that there is no disadvantage in the second scenario: Alice and Bob, without knowing the other party's string, can compress their strings to almost minimal description length in the sense of Kolmogorov complexity. Furthermore, compression takes polynomial time and can be made at any combination of lengths that satisfy some necessary conditions (modulo additive polylogarithmic terms). More precisely, there exist probabilistic algorithms E 1 , E 2 , and deterministic algorithm D, with E 1 and E 2 running in polynomial time, having the following behavior: if n 1 , n 2 are two integers satisfying n 1

show abstract

“…The following is the particular case of the above theorem for ℓ = 1. A weaker version appeared in reference [BZ14].…”

mentioning

confidence: 99%

An Operational Characterization of Mutual Information in Algorithmic Information Theory

Romashchenko

Zimand

2019

J. ACM

Self Cite

View full text Add to dashboard Cite

We show that the mutual information, in the sense of Kolmogorov complexity, of any pair of strings x and y is equal, up to logarithmic precision, to the length of the longest shared secret key that two parties, one having x and the complexity profile of the pair and the other one having y and the complexity profile of the pair, can establish via a probabilistic protocol with interaction on a public channel. For ℓ > 2, the longest shared secret that can be established from a tuple of strings (x1, ..., x ℓ ) by ℓ parties, each one having one component of the tuple and the complexity profile of the tuple, is equal, up to logarithmic precision, to the complexity of the tuple minus the minimum communication necessary for distributing the tuple to all parties. We establish the communication complexity of secret key agreement protocols that produce a secret key of maximal length, for protocols with public randomness. We also show that if the communication complexity drops below the established threshold, then only very short secret keys can be obtained. supported in part by the National Science Foundation through grant 1811729. No protocol can produce a longer shared secret key (up to an O(log n) additive term).Secret key agreement for three or more parties. Mutual information is only defined for two strings, but secret key agreement can be explored for the case of more strings. Let us consider again an example. Suppose that each of Alice, Bob, and Charlie have a point in the affine plane over the finite field with 2 n elements, and that the three points, which we call A, B, C, are collinear. Thus, each party has 2n bits of information, but together they have 5n bits of information, because given two points, the third one can be described with n bits. The parties want to establish a common secret key, but they can only communicate by broadcasting messages over a public channel. They can proceed as follows. Alice will broadcast a string p A , Bob a string p B , and Charlie a string p C , such that each party using his/her point and the received information will reconstruct the three collinear points A, B, C. A protocol that achieves this is called an omniscience protocol because it spreads to everyone the information possessed at the beginning individually by each party. In the next step, each party will compress the 5n bits, comprising the three points, to a string that is random given p A , p B , p C . The compressed string is the common secret key. We will see that up to logarithmic precision it has length 5n − (|p A | + |p B | + |p C |). Assuming we know how to do the omniscience protocol and the compression step, this protocol produces a common secret key of length 5n − CO(A, B, C), where CO (A, B, C) is the minimum communication for the omniscience task for the points A, B, C. In our example, it is clear that each one of p A , p B , p C must be at least n bits long, and that any two of these strings must contain together at least 3n bits. Using some recent results from the reference [Zim17], it can be shown that an...

show abstract

Linear List-Approximation for Short Programs (or the Power of a Few Random Bits)

Cited by 15 publications

References 16 publications

Compression of Data Streams Down to Their Information Content

Compression of Data Streams Down to Their Information Content

Kolmogorov complexity version of Slepian-Wolf coding

An Operational Characterization of Mutual Information in Algorithmic Information Theory

Contact Info

Product

Resources

About