$LT^2C^2$ : A language of thought with Turing-computable Kolmogorov complexity

Abstract

In this paper, we present a theoretical effort to connect the theory of program size to psychology by implementing a concrete language of thought with Turing-computable Kolmogorov complexity (LT 2 C 2 ) satisfying the following requirements: 1) to be simple enough so that the complexity of any given finite binary sequence can be computed, 2) to be based on tangible operations of human reasoning (printing, repeating,. . . ), 3) to be sufficiently powerful to generate all possible sequences but not too powerful as to identify regularities which would be invisible to humans. We first formalize LT 2 C 2 , giving its syntax and semantics and defining an adequate notion of program size. Our setting leads to a Kolmogorov complexity function relative to LT 2 C 2 which is computable in polynomial time, and it also induces a prediction algorithm in the spirit of Solomonoff's inductive inference theory. We then prove the efficacy of this language by investigating regularities in strings produced by participants attempting to generate random strings. Participants had a profound understanding of randomness and hence avoided typical misconceptions such as exaggerating the number of alternations. We reasoned that remaining regularities would express the algorithmic nature of human thoughts, revealed in the form of specific patterns. Kolmogorov complexity relative to LT 2 C 2 passed three expected tests examined here: 1) human sequences were less complex than control PRNG sequences, 2) human sequences were not stationary, showing decreasing values of complexity resulting from fatigue, 3) each individual showed traces of algorithmic stability since fitting of partial sequences was more effective to predict subsequent sequences than average fits. This work extends on previous efforts to combine notions of Kolmogorov complexity theory and algorithmic information theory to psychology, by explicitly proposing a language which may describe the patterns of human thoughts. *

Mental compression of binary sequences in a language of thought

“…However, KC can be approximated in a computable way, and one way to do it is by considering domain-specific languages that encode a finite set of relevant function (i.e. a semantics) (Romano, Sigman, & Figueira, 2013). Recently, the group of Gauvrit, Delahaye, Zenil and Soler-Toscano proposed an approximation to KC using the coding theorem as starting point; this theorem relates the algorithmic complexity of a sequence to the probability that a universal machine outputs such MENTAL COMPRESSION OF BINARY SEQUENCES 6 sequence (Delahaye & Zenil, 2012;Gauvrit, Singmann, Soler-Toscano, & Zenil, 2016;.…”

“…As a result, the language favors a description of sequence based on nested repetitions of instructions, thus sharply dissociating sequence length and complexity. Consistent with the "minimal description length" principle, among the multiple expressions that can describe the same sequence, the expression with the lowest complexity is considered to correspond to the human mental representation of the sequence (and its length approximates the Kolmogorov complexity of the sequence; see Grunwald, 2004;Romano et al, 2013). In a nutshell, the assumption is that, in order to minimize memory load, participants mentally compress the sequence structure using the proposed formal language.…”

Bayesian selection of grammar productions for the language of thought

“…The LoT is not necessarily unique. In fact, the form that it takes has been modeled in many different ways depending on the problem domain: nu-merical concept learning (Piantadosi, Tenenbaum, & Goodman, 2012), sequence learning (Amalric et al, 2017;Romano, Sigman, & Figueira, 2013;Yildirim & Jacobs, 2015), visual concept learning (Ellis, Solar-Lezama, & Tenenbaum, 2015), theory learning (Ullman, Goodman, & Tenenbaum, 2012), etc.…”

“…Most studies of LoTs have focused on the compositional aspect of the language, which has either been modeled within a Bayesian (Tenenbaum et al, 2011) or a Minimum Description Length (MDL) framework (Amalric et al, 2017;Goldsmith, 2001Goldsmith, , 2002Romano et al, 2013).…”

“…While frameworks may differ on how a LoT may be implemented computationally, they all share the property of being built from a set of atomic symbols and rules by which they can be combined to form new and more complex expressions. Most studies of LoTs have focused on the compositional aspect of the language, which has either been modeled within a Bayesian (Tenenbaum et al, 2011) or a Minimum Description Length (MDL) framework (Amalric et al, 2017;Goldsmith, 2001Goldsmith, , 2002Romano et al, 2013). The common method is to define a grammar with a set of productions based on operations that are intuitive to researchers and then study how different inference processes match regular patterns in human learning.…”