Typology emerges from simplicity in representations and learning

Abstract

We derive well-understood and well-studied subregular classes of formal languages purely from the computational perspective of algorithmic learning problems. We parameterise the learning problem along dimensions of representation and inference strategy. Of special interest are those classes of languages whose learning algorithms are necessarily not prohibitively expensive in space and time, since learners are often exposed to adverse conditions and sparse data. Learned natural language patterns are expected to be most like the patterns in these classes, an expectation supported by previous typological and linguistic research in phonology. A second result is that the learning algorithms presented here are completely agnostic to choice of linguistic representation. In the case of the subregular classes, the results fall out from traditional model-theoretic treatments of words and strings. The same learning algorithms, however, can be applied to model-theoretic treatments of other linguistic representations such as syntactic trees or autosegmental graphs, which opens a useful direction for future research.

Natural language syntax complies with the free-energy principle

“…The restrictiveness generated by these complexity measures supplements the more general simplicity criterion of theoretical syntax; much as how the 'subset principle' (restrictiveness) supplemented the original evaluation metric (simplicity) (Berwick, 1985). Lambert et al (2021) demonstrated that the computational simplicity of learning mechanisms appears to have a major impact on the types of patterns found in natural language, including for syntactic trees, and so it seems to us well motivated to turn to the issue of the underlying processes that guide the generation of these structures.…”

Robust Identification in the Limit from Incomplete Positive Data

“…Many other subregular classes exist, but only these few will be discussed in this work. For more details, readers are referred to Lambert et al [19]. SF is star-free [20].…”

Natural language syntax complies with the free-energy principle

“…The restrictiveness generated by these complexity measures supplements the more general simplicity criterion of theoretical syntax; much as how the 'subset principle' (restrictiveness) supplemented the original evaluation metric (simplicity) (Berwick 1985). Lambert et al (2021) demonstrated that the computational simplicity of learning mechanisms appears to have a major impact on the types of patterns found in natural language, including for syntactic trees, and 26 so it seems to us well motivated to turn to the issue of the underlying processes that guide the generation of these structures.…”