Approximate Piecewise Affine Decomposition of Neural Networks

Robinson, Haakon

doi:10.1016/j.ifacol.2021.08.416

Cited by 2 publications

(1 citation statement)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In artificial intelligence, ReLU networks can be characterized by the conjunction of a set of linear inequalities which define a polytope in the input domain known as the activation condition [9]. In a related vein, the regularization of neural networks using piecewise affine functions involves the evaluation of the volume of polytopes defined by intersections of these hyperplanes [10].…”

Section: Volume Of Polytopesmentioning

confidence: 99%

Efficient computation of the volume of a polytope in high-dimensions using Piecewise Deterministic Markov Processes

Chevallier¹,

Cazals²,

Fearnhead³

2022

Preprint

View full text Add to dashboard Cite

Computing the volume of a polytope in high dimensions is computationally challenging but has wide applications. Current state-of-the-art algorithms to compute such volumes rely on efficient sampling of a Gaussian distribution restricted to the polytope, using e.g. Hamiltonian Monte Carlo. We present a new sampling strategy that uses a Piecewise Deterministic Markov Process. Like Hamiltonian Monte Carlo, this new method involves simulating trajectories of a non-reversible process and inherits similar good mixing properties. However, importantly, the process can be simulated more easily due to its piecewise linear trajectories -and this leads to a reduction of the computational cost by a factor of the dimension of the space. Our experiments indicate that our method is numerically robust and is one order of magnitude faster (or better) than existing methods using Hamiltonian Monte Carlo. On a single core processor, we report computational time of a few minutes up to dimension 500.

show abstract

Section: Volume Of Polytopesmentioning

confidence: 99%