Deep Neural Network Approximation For Hölder Functions

Abstract

In this work, we explore the approximation capability of deep Rectified Quadratic Unit neural networks for Hölder-regular functions, with respect to the uniform norm. We find that theoretical approximation heavily depends on the selected activation function in the neural network.