Tropical Geometry of Deep Neural Networks

Zhang, Liwen; Naitzat, Gregory; Lim, Lek-Heng

doi:10.48550/arxiv.1805.07091

Cited by 10 publications

(14 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…respectively. We may associate a function ν : R d → R p , where d = n 1 and p = m L to the neural network (see [9]) as follows…”

Section: Neural Networkmentioning

confidence: 99%

“…The other being Zhang et. al [9], which draws an explicit connection between tropical rational functions and feed-forward neural networks with piecewise linear activation functions.…”

Section: Introductionmentioning

confidence: 99%

“…Unlike in some other sources what we mean by a neural network is the data of weights, biases, and threshold vectors on a neural architecture i.e., a directed acyclic graph. We associate a function to the neural network by composing the activation functions as done in say [9]. We say that two neural networks are equivalent if their associated functions are identical.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

On the space of coefficients of a Feed Forward Neural Network

Valluri¹,

Campbell²

2021

Preprint

View full text Add to dashboard Cite

We define and establish the conditions for 'equivalent neural networks' -neural networks with different weights, biases, and threshold functions which result in the same associated function. We prove that, given a neural network N with piecewise linear activation, the space of coefficients describing all equivalent neural networks is given by a semialgebraic set. This result is obtained by studying different representations of a given piece-wise linear function using the Tarski-Seidenberg theorem.

show abstract

“…respectively. We may associate a function ν : R d → R p , where d = n 1 and p = m L to the neural network (see [9]) as follows…”

Section: Neural Networkmentioning

confidence: 99%

“…The other being Zhang et. al [9], which draws an explicit connection between tropical rational functions and feed-forward neural networks with piecewise linear activation functions.…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On the space of coefficients of a Feed Forward Neural Network

Valluri¹,

Campbell²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Zhang et al [22] showed that due to the piecewise linear structure of these neural networks, and under certain assumptions, the set of ReLU neural networks, the set of piecewise linear functions, and the set of tropical rational functions are equivalent. We do not extend our results to the realm of tropical algebra, but we do take inspiration from the concept of the dual as commonly expressed in tropical algebra.…”

mentioning

confidence: 99%

“…Linear Regions. Neural networks with piecewise linear activation functions, such as ReLU, are continuous piecewise linear maps from the input space to the output space [22]. Additionally, each of the linear portions of this mapping is supported on a convex polytope.…”

mentioning

confidence: 99%

Locally Linear Attributes of ReLU Neural Networks

Sattelberg,

Cavalieri,

Kirby

et al. 2020

Preprint

View full text Add to dashboard Cite

A ReLU neural network determines/is a continuous piecewise linear map from an input space to an output space. The weights in the neural network determine a decomposition of the input space into convex polytopes and on each of these polytopes the network can be described by a single affine mapping. The structure of the decomposition, together with the affine map attached to each polytope, can be analyzed to investigate the behavior of the associated neural network.

show abstract

Some open questions in quiver gauge theory

Bao

Hanany

et al. 2022

Proyecciones (Antofagasta)

View full text Add to dashboard Cite

Quivers, gauge theories and singular geometries are of great interest in both mathematics and physics. In this note, we collect a few open questions which have arisen in various recent works at the intersection between gauge theories, representation theory, and algebraic geometry. The questions originate from the study of supersymmetric gauge theories in different dimensions with different supersymmetries. Although these constitute merely the tip of a vast iceberg, we hope this guide can give a hint of possible directions in future research.

show abstract

Tropical Geometry of Deep Neural Networks

Cited by 10 publications

References 0 publications

On the space of coefficients of a Feed Forward Neural Network

On the space of coefficients of a Feed Forward Neural Network

Locally Linear Attributes of ReLU Neural Networks

Some open questions in quiver gauge theory

Contact Info

Product

Resources

About