The Computational Power of Interactive Recurrent Neural Networks

Abstract

In classical computation, rational-and real-weighted recurrent neural networks were shown to be respectively equivalent to and strictly more powerful than the standard Turing machine model. Here, we study the computational power of recurrent neural networks in a more biologically-oriented computational framework capturing the aspects of sequential interactivity and persistence of memory. In this context, we prove that so-called interactive rational-and real-weighted neural networks show the same computational powers as interactive Turing machines and interactive Turing machines with advice, respectively. A mathematical characterization of each of these computational powers is also provided. It follows from these results that interactive real-weighted neural networks can actually perform uncountably many more translations of information than interactive Turing machines, making them capable of super-Turing capabilities.

Recurrent Neural Networks and Super-Turing Interactive Computation

“…In fact, the I-TMs and I-TM/As realize precisely the classes of recursive continuous and continuous ω-translations, respectively. The following results are proven in [11]. Since these proofs can be easily deduced from those of previous Proposition 1 and forthcoming Lemma 1, we can include them hereafter.…”

“…They further introduced the concept of interactive Turing machine with advice (I-TM/A) as a relevant non-uniform computational model in the context of interactive computation [69,70]. Interactive Turing machines with advice were proven to be strictly more powerful than interactive Turing machines without advice [70,Proposition 5] and [69,Lemma 1], and were shown to be computationally equivalent to several other non-uniform models of interactive computation, like sequences of interactive finite automata, site machines, web Turing machines [69,70], and more recently to interactive analog neural networks and interactive evolving neural networks [9,11,14].…”

“…More recently, Cabessa and Siegelmann studied the computational power of RNNs involved in an interactive computational framework similar the one presented here. First, they proved that interactive rational RNNs are Turing-equivalent, and hence realize the class of recursive continuous ω-translations [11]. The results is formally expressed as follows.…”

“…Cabessa and Villa also provided a description of the super-Turing computational power of analog recurrent neural networks engaged in a similar reactive computational scenario [13]. Besides, Cabessa and Siegelmann provided a characterization of the Turing and super-Turing capabilities of rational and analog recurrent neural networks involved in a more bio-inspired interactive computational paradigm [11].…”

What kind of machine is the mind?

“…As an aside, if the link weights were real-valued (i.e. if they had infinite precision), ASMs would be more powerful than TMs, i.e., hypercomputational (Cabessa & Siegelmann, 2012). This is of no relevance, however, because the transfer function would no longer be efficiently computable (just as the physical universe does not afford an infinite resolution to which computational systems can be realized).…”