Characteristics of discrete transfinite time Turing machine models: Halting times, stabilization times, and Normal Form theorems

Abstract

The authors present empirical distributions for the halting time (measured by the number of iterations to reach a given accuracy) of optimization algorithms applied to two random systems: spin glasses and deep learning. Given an algorithm, which we take to be both the optimization routine and the form of the random landscape, the fluctuations of the halting time follow a distribution that, after centering and scaling, remains unchanged even when the distribution on the landscape is changed. We observe two qualitative classes: A Gumbel-like distribution that appears in Google searches, human decision times, the QR eigenvalue algorithm and spin glasses, and a Gaussian-like distribution that appears in conjugate gradient method, deep network with MNIST input data and deep network with random input data. This empirical evidence suggests presence of a class of distributions for which the halting time is independent of the underlying distribution under some conditions.

Feedback Turing Computability, and Turing Computability as Feedback

“…What makes this plausible is the connection on both sides with Σ 2 reflection. Welch [14], [15] characterized the closure point of regular ITTMs via L ζ , the least initial segment of L with a Σ 2 elementary extension L Σ . So you would expect FITTMs to have a closure point at some sort of super-Σ 2 reflecting ordinal.…”

Transfinite machine models

“…[77], [83]) (i) Any ITTM computation P e (x) which halts, does so by time λ x , the latter being defined as the supremum of the x-writable ordinals; (ii) any computation P e (x) with eventually stable output tape, will stabilize before the time ζ x defined as the supremum of the eventually x-writable ordinals; (iii) moreover ζ x is the least ordinal so that there exists Σ x > ζ x with the property that…”

“…[28] showed that for almost all f , N P f = P f and later [14] showed that for many α P α = N P α ∩ co-N P α including those α that begin a gap in the clockable ordinals. (Such are admissible by [83] Thm. 50).…”

Discrete Transfinite Computation Models

“…The answer turns out, thankfully, to be affirmative, (it follows from the λ, ζ, Σ-Theorem below, [Welch (2000)]). We refer the reader not to the original papers, but to [Welch (2009)] for a later but somewhat tidier account of this theorem and the answers to the above three questions.…”

“…Theorem 1.2. (Normal Form Theorem I [Welch (2004)], [Welch (2009)]) ∀e∃e ∀n ∈ N P e (n) ↓−→ (P e (n) ↓ y where y ∈ 2 N codes a wellordered course-ofcomputation sequence for P e (x) ↓).…”