Algorithmic Information Theory for Physicists and Natural Scientists

Devine, Sean

doi:10.1088/978-0-7503-2640-7

Cited by 7 publications

(21 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is, however, another branch of information theory, called algorithmic information theory (AIT) 4 , which is concerned with the information content of individual objects. It has been much less applied in physics (although notable exceptions occur, see 5 for a recent overview). Reasons for this relative lack of attention include that AIT's central concept, the Kolmogorov complexity K U (x) of a string x, defined as the length of the shortest program that generates x on an optimal reference universal Turing machine (UTM) U, is formally uncomputable due to its link to the famous halting problem of UTMs 6 -see 7 for technical details.…”

Section: Generic Predictions Of Output Probability Based On Complexitmentioning

confidence: 99%

Generic predictions of output probability based on complexities of inputs and outputs

Dingle

Pérez

Louis

2020

Sci Rep

View full text Add to dashboard Cite

For a broad class of input-output maps, arguments based on the coding theorem from algorithmic information theory (AIT) predict that simple (low Kolmogorov complexity) outputs are exponentially more likely to occur upon uniform random sampling of inputs than complex outputs are. Here, we derive probability bounds that are based on the complexities of the inputs as well as the outputs, rather than just on the complexities of the outputs. The more that outputs deviate from the coding theorem bound, the lower the complexity of their inputs. Our new bounds are tested for an RNA sequence to structure map, a finite state transducer and a perceptron. These results open avenues for AIT to be more widely used in physics. PACS numbers:Deep links between physics and theories of computation [1, 2] are being increasingly exploited to uncover new fundamental physics and to provide novel insights into theories of computation. For example, advances in understanding quantum entanglement are often expressed in sophisticated information theoretic language, while providing new results in computational complexity theory such as polynomial time algorithms for integer factorization [3]. These connections are typically expressed in terms of Shannon information, with its natural analogy with thermodynamic entropy.There is, however, another branch of information theory, called algorithmic information theory (AIT) [4], which is concerned with the information content of individual objects. It has been much less applied in physics (although notable exceptions occur, see [5] for a recent overview). Reasons for this relative lack of attention include that AIT's central concept, the Kolmogorov complexity K U (x) of a string x, defined as the length of the shortest program that generates x on a universal Turing machine (UTM) U , is formally uncomputable due to its link to the famous halting problem of UTMs [6]. Moreover, many important results, such as the invariance theorem which states that for two UTMs U and W , the Kolmogorov complexities K U (x) = K W (x) + O(1) are equivalent, hold asymptotically up to O(1) terms that are independent of x, but not always well understood, and therefore hard to control.Another reason applications of AIT to many practical problems have been hindered can be understood in terms of hierarchies of computing power. For example, one of the oldest such categorisations, the Chomsky hierarchy [7], ranks automata into four different classes, of which the UTMs are the most powerful, and finite state machines (FSMs) are the least. Many key results in AIT are derived by exploiting the power of UTMs. Interestingly, if physical processes can be mapped onto UTMs, then certain properties can be shown to be uncom-putable [8, 9]. However, many problems in physics are fully computable, and therefore lower on the Chomsky hierarchy than UTMs. For example, finite Markov processes are equivalent to FSMs, and RNA secondary structure (SS) folding algorithms can be recast as context-free grammars, the second level in the hiearchy. Th...

show abstract

Section: Generic Predictions Of Output Probability Based On Complexitmentioning

confidence: 99%

Generic predictions of output probability based on complexities of inputs and outputs

Dingle

Pérez

Louis

2020

Sci Rep

View full text Add to dashboard Cite

show abstract

“…The natural laws that drive real-world behaviour are in effect computations on a universal computer known as a Universal Turing Machine (UTM). The energy states of chemical species with their rules of interactions are the equivalent of gates in a conventional computer [ 16 , 17 ]. However, as one UTM can simulate another to within a constant, a laboratory reference UTM, using binary coding, can in principle simulate the natural computations implementing physical laws in the real-world.…”

Section: Information and Ordermentioning

confidence: 99%

“…Ayres [ 24 ] Equation (10), articulates a similar argument for the Shannon entropy of a system consisting of n compartments. He shows that the Shannon entropy is equal to the entropy (which here would be the programme bits) in, less the entropy out, plus the entropy captured in the structure, i.e., the net flow of bits is identical to the number of bits in the length of the shortest algorithm that halts after generating a snapshot of the system’s configuration at a particular instant [ 17 ].…”

Section: Information and Ordermentioning

confidence: 99%

An Economy Viewed as a Far-from-Equilibrium System from the Perspective of Algorithmic Information Theory

Devine

2018

Entropy

View full text Add to dashboard Cite

This paper, using Algorithmic Information Theory (AIT), argues that once energy resources are considered, an economy, like an ecology, requires continuous energy to be sustained in a homeostatic state away from the decayed state of its (local) thermodynamic equilibrium. AIT identifies how economic actions and natural laws create an ordered economy through what is seen as computations on a real world Universal Turing Machine (UTM) that can be simulated to within a constant on a laboratory UTM. The shortest, appropriately coded, programme to do this defines the system's information or algorithmic entropy. The computational behaviour of many generations of primitive economic agents can create a more ordered and advanced economy, able to be specified by a relatively short algorithm. The approach allows information flows to be tracked in real-world computational processes where instructions carried in stored energy create order while ejecting disorder. Selection processes implement the Maximum Power Principle while the economy trends towards Maximum Entropy Production, as tools amplify human labour and interconnections create energy efficiency. The approach provides insights into how an advanced economy is a more ordered economy, and tools to investigate the concerns of the Bioeconomists over long term economic survival.

show abstract

“…Later the argument tightened when erasure was seen to be the reduction in the number of bits specifying a microstate of a system and, for a quantum system, the change in its Von Neumann entropy as discussed in Section 5. Nevertheless, for these reasons it is unclear how valid the principle is for regimes where the assumptions do not hold, particularly as Devine [3,4] used the principle to identify the thermodynamic cost of maintaining a system distant from equilibrium in terms of entropy flows.…”

Section: Introductionmentioning

confidence: 99%

“…As the argument applies to systems distant from equilibrium, it justifies the use of Landauer's principle to track bit flows into and out of the non-equilibrium system as used by Devine [3,4]. By being able to do so, AIT offers an approach to statistical thermodynamics which is easy to visualize and which offers new insights, as these computational bits are real entities linked to energy through the temperature.…”

Section: Introductionmentioning

confidence: 99%

Landauer’s Principle a Consequence of Bit Flows, Given Stirling’s Approximation

Devine

2021

Entropy

View full text Add to dashboard Cite

According to Landauer’s principle, at least kBln2T Joules are needed to erase a bit that stores information in a thermodynamic system at temperature T. However, the arguments for the principle rely on a regime where the equipartition principle holds. This paper, by exploring a simple model of a thermodynamic system using algorithmic information theory, shows the energy cost of transferring a bit, or restoring the original state, is kBln2T Joules for a reversible system. The principle is a direct consequence of the statistics required to allocate energy between stored energy states and thermal states, and applies outside the validity of the equipartition principle. As the thermodynamic entropy of a system coincides with the algorithmic entropy of a typical state specifying the momentum degrees of freedom, it can quantify the thermodynamic requirements in terms of bit flows to maintain a system distant from the equilibrium set of states. The approach offers a simple conceptual understanding of entropy, while avoiding problems with the statistical mechanic’s approach to the second law of thermodynamics. Furthermore, the classical articulation of the principle can be used to derive the low temperature heat capacities, and is consistent with the quantum version of the principle.

show abstract

Algorithmic Information Theory for Physicists and Natural Scientists

Cited by 7 publications

References 65 publications

Generic predictions of output probability based on complexities of inputs and outputs

Generic predictions of output probability based on complexities of inputs and outputs

An Economy Viewed as a Far-from-Equilibrium System from the Perspective of Algorithmic Information Theory

Landauer’s Principle a Consequence of Bit Flows, Given Stirling’s Approximation

Contact Info

Product

Resources

About