New neural networks based on Taylor series and their research

Chen, Xiaoyun; Ma, Qiang; Al-Kharobi, Talal

doi:10.1109/iccsit.2009.5234726

Cited by 6 publications

(5 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As to the TaylorNet, it is also worth mentioning that our proposal is very different from the previous work in the literature [91][92][93][94][95], which have also explored the Taylor series in neural networks. Most of them have specific motivations and work in different frameworks, and there is no explicit hierarchical structure employed there.…”

Section: Discussionmentioning

confidence: 95%

Exploring explicit coarse-grained structure in artificial neural networks

Xi-Ci¹,

Xie²,

Yang³

2022

Preprint

View full text Add to dashboard Cite

We propose to employ the hierarchical coarse-grained structure in the artificial neural networks explicitly to improve the interpretability without degrading performance. The idea has been applied in two situations. One is a neural network called TaylorNet, which aims to approximate the general mapping from input data to output result in terms of Taylor series directly, without resorting to any magic nonlinear activations. The other is a new setup for data distillation, which can perform multilevel abstraction of the input dataset and generate new data that possesses the relevant features of the original dataset and can be used as references for classification. In both cases, the coarse-grained structure plays an important role in simplifying the network and improving both the interpretability and efficiency. The validity has been demonstrated on MNIST and CIFAR-10 datasets. Further improvement and some open questions related are also discussed.

show abstract

Section: Discussionmentioning

confidence: 95%

Exploring explicit coarse-grained structure in artificial neural networks

Xi-Ci¹,

Xie²,

Yang³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Take the Taylor series of the unary function as an example. In order to describe the finite Taylor series, considering the sum of the first + 1 terms, we can get formula (5).…”

Section: Neural Network Modelmentioning

confidence: 99%

“…Explicit polynomial fitting often uses Taylor series [5,6]. Taylor series are often used in the field of mathematics, especially in the research of approximate calculations.…”

Section: Introductionmentioning

confidence: 99%

Polynomial Fitting Algorithm Based on Neural Network

Tong

et al. 2021

ASP Trans. Pattern Recognit. Intell. Syst.

View full text Add to dashboard Cite

As a method of function approximation, polynomial fitting has always been the main research hotspot in mathematical modeling. In many disciplines such as computer, physics, biology, neural networks have been widely used, and most of the applications have been transformed into fitting problems using neural networks. One of the main reasons that neural networks can be widely used is that it has a certain sense of universal approximation. In order to fit the polynomial, this paper constructs a three-layer feedforward neural network, uses Taylor series as the activation function, and determines the number of hidden layer neurons according to the order of the polynomial and the dimensions of the input variables. For explicit polynomial fitting, this paper uses non-linear functions as the objective function, and compares the fitting effects under different orders of polynomials. For the fitting of implicit polynomial curves, the current popular polynomial fitting algorithms are compared and analyzed. Experiments have proved that the algorithm used in this paper is suitable for both explicit polynomial fitting and implicit polynomial fitting. The algorithm is relatively simple, practical, easy to calculate, and can efficiently achieve the fitting goal. At the same time, the computational complexity is relatively low, which has certain application value.

show abstract

“…As to the TaylorNet, it is also worth mentioning that our proposal is very different from the previous work in the literature, [91][92][93][94][95][96][97] which have also explored the Taylor series in neural networks. Most of them have specific motivations and work in different frameworks, and there is no explicit hierarchical structure employed there.…”

mentioning

confidence: 96%

“…Most of them have specific motivations and work in different frameworks, and there is no explicit hierarchical structure employed there. For example, Chen et al [91] used a single-layer neural network similar to Eq. ( 3) to approximate the Taylor expansion of a single-variable function.…”

mentioning

confidence: 99%

Exploring Explicit Coarse-Grained Structure in Artificial Neural Networks

Xi-Ci

Xie

Yang

2023

Chinese Phys. Lett.

View full text Add to dashboard Cite

We propose to employ the hierarchical coarse-grained structure in the artificial neural networks explicitly to improve the interpretability without degrading performance. The idea has been applied in two situations. One is a neural network called TaylorNet, which aims to approximate the general mapping from input data to output result in terms of Taylor series directly, without resorting to any magic nonlinear activations. The other is a new setup for data distillation, which can perform multi-level abstraction of the input dataset and generate new data that possesses the relevant features of the original dataset and can be used as references for classification. In both cases, the coarse-grained structure plays an important role in simplifying the network and improving both the interpretability and efficiency. The validity has been demonstrated on MNIST and CIFAR-10 datasets. Further improvement and some open questions related are also discussed.

show abstract

New neural networks based on Taylor series and their research

Cited by 6 publications

References 2 publications

Exploring explicit coarse-grained structure in artificial neural networks

Exploring explicit coarse-grained structure in artificial neural networks

Polynomial Fitting Algorithm Based on Neural Network

Exploring Explicit Coarse-Grained Structure in Artificial Neural Networks

Contact Info

Product

Resources

About