A Logical Characterization of Constant-Depth Circuits over the Reals

Abstract

We characterize the computational power of neural networks that follow the graph neural network (GNN) architecture, not restricted to aggregatecombine GNNs or other particular types. We establish an exact correspondence between the expressivity of GNNs using diverse activation functions and arithmetic circuits over real numbers. In our results the activation function of the network becomes a gate type in the circuit. Our result holds for families of constant depth circuits and networks, both uniformly and non-uniformly, for all common activation functions.

Logical characterizations of algebraic circuit classes over integral domains

“…For future work, it would be interesting to investigate the logical characterizations made in this paper in the uniform setting. We know that for the real numbers, the characterization AC 0 R = FO R [Arb R ] + SUM R + PROD R holds both non-uniformly and for uniformity criteria given by polynomial time computable circuits (P R uniform), logarithmic time computable circuits (LT R uniform) and first-order definable circuits (FO R uniform) (Barlag and Vollmer, 2021). We believe that the results we presented here hold in analogous uniform settings as well, though this would need to be further examined.…”

“…Proof. The proof for this theorem works similarly to the construction for FO R and Vollmer, 2021), since this construction does not make use of any special properties of the real numbers.…”

“…The generalization to algebraic circuits over real numbers were first introduced by Cucker (1992). In analogy to them, Barlag and Vollmer defined an unbounded fan-in version of algebraic circuits (Barlag and Vollmer, 2021). Definition 8.…”

“…Remark 9. We will use the term algebraic circuit to describe circuits in the sense of Blum et al (1998) rather than arithmetic circuits as for example in Barlag and Vollmer (2021) to distinguish them from arithmetic circuits in the sense of Valiant, see for example (Bürgisser et al, 1997). Valiant circuits are essentially algebraic circuits without sign or comparison gates (Blum et al, 1998, page 350).…”

“…Remark 12. Unlike the way algebraic circuits with unbounded fan-in gates were introduced previously (Barlag and Vollmer, 2021), algebraic circuits in this context have comparison gates instead of sign gates. This stems from the fact that when dealing with real or complex numbers, we can construct a sign function from < via Definition 7 and the order relation from the sign function via…”