Training Neural Networks is $\exists\mathbb R$-complete

Abstract

Given a neural network, training data, and a threshold, it was known that it is NP-hard to find weights for the neural network such that the total error is below the threshold. We determine the algorithmic complexity of this fundamental problem precisely, by showing that it is ∃R-complete. This means that the problem is equivalent, up to polynomial time reductions, to deciding whether a system of polynomial equations and inequalities with integer coefficients and real unknowns has a solution. If, as widely expected, ∃R is strictly larger than NP, our work implies that the problem of training neural networks is not even in NP.

Topological Art in Simple Galleries

“…If one of the guards of component (3) is at the left endpoint of its segment, it guards all four circle pockets. The left and right parts of Figure 34 show the two configurations of the guards in component (3) for which this holds. We refer to them as unlocking configurations, and the rest of the configurations of the guards in component (3) are referred to as locking configurations.…”

“…, X n ]. As we will not use ∃R-completeness, except for pointing out its link to homotopyuniversality, we merely refer to some surveys and recent developments [3,13,19,29,31,34,35].…”

“…Lemma 26. The solution space, restricted to solutions in which the guards of component (3) are in some locking configuration, is homeomorphic to a circle S 1 .…”

“…Figure 36 shows these possible movements. One can see that the solution space is homeomorphic to a circle Figure 36: Solution space formed by the guards of components (1) and (2) when the guards of component (3) are in some fixed locking configuration. One of the guards has to see the hole pocket, its visibility area is indicated in gray.…”

The Complexity of the Hausdorff Distance

“…Famous examples from discrete geometry are the recognition of geometric structures, such as unit disk graphs [34], segment intersection graphs [33], visibility graphs [20], stretchability of pseudoline arrangements [37,49], and order type realizability [33]. Other ∃R-complete problems are related to graph drawing [32], Nash-Equilibria [15,28], geometric packing [6], the art gallery problem [3], convex covers [2], non-negative matrix factorization [48], polytopes [25,42], geometric embeddings of simplicial complexes [4], geometric linkage constructions [1], training neural networks [5], and continuous constraint satisfaction problems [35]. We refer the reader to the lecture notes by Matoušek [33] and surveys by Schaefer [45] and Cardinal [19] for more information on the complexity class ∃R.…”