Applications of classical approximation theory to periodic basis function networks and computational harmonic analysis

Mhaskar, H. N.; Nevai, Paul; Shvarts, Eugene

doi:10.1007/s13373-013-0043-1

Cited by 6 publications

(3 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…K to guarantee that the estimator we construct is agnostic to the unknown support size S. In practice, u K and v K can be replaced by the efficiently implementable lowpass filtered Chebyshev expansion [19], which achieves the same uniform error rate as the best polynomial approximation. Remark 2.…”

Section: B Construction Of the Optimal Estimatormentioning

confidence: 99%

Minimax estimation of the L<inf>1</inf> distance

Jiao

Han

Weissman

2016

2016 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

We consider the problem of estimating the L1 distance between two discrete probability measures P and Q from empirical data in a nonasymptotic and large alphabet setting. When Q is known and one obtains n samples from P , we show that for every Q, the minimax rate-optimal estimator with n samples achieves performance comparable to that of the maximum likelihood estimator (MLE) with n ln n samples. When both P and Q are unknown, we construct minimax rateoptimal estimators whose worst case performance is essentially that of the known Q case with Q being uniform, implying that Q being uniform is essentially the most difficult case. The effective sample size enlargement phenomenon, identified in Jiao et al. (2015), holds both in the known Q case for every Q and the Q unknown case. However, the construction of optimal estimators for P − Q 1 requires new techniques and insights beyond the approximation-based method of functional estimation in .

show abstract

Section: B Construction Of the Optimal Estimatormentioning

confidence: 99%

Minimax estimation of the L<inf>1</inf> distance

Jiao

Han

Weissman

2016

2016 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

show abstract

“…In particular, by noticing that the expansion in Eq. (4) represents an ( n – 1)-th order polynomial expression in A , we can re-interpret the reconstruction algorithm as a high-dimensional polynomial approximation algorithm or a neural network 20,21 . That is, we can generalize Eq.…”

Section: Methodsmentioning

confidence: 99%

Nonlinear electrocardiographic imaging using polynomial approximation networks

et al. 2018

View full text Add to dashboard Cite

Electrocardiography is a valuable tool to aid in medical understanding and treatment of heart-related ailments, specifically atrial fibrillation (AF) and other irregular cardiac behavior. Although signs of AF will manifest in conventional electrocardiogram (ECG) recordings, interpretation and localization of AF sources require significant clinical expertise. In this vein, electrocardiographic imaging has emerged as an important medical imaging modality that provides reconstructions of the heart's electrical activity from non-invasive multi-lead body-surface ECG and anatomical x-ray computed tomography images. In this paper, we present a nonlinear inversion model for computing this mapping to improve upon the reconstruction performance of current methods. While contemporary techniques typically determine an inverse solution by discretizing and inverting an underdetermined linear system of partial differential equations governing the relationship between voltage potentials of the heart and torso, the presented technique re-casts this problem as a task in function approximation and provides a direct parameterization of the inverse operator using a polynomial neural network. That is, the outlined nonlinear inversion technique is a generalization of contemporary reconstruction techniques which allows geometrical and material parameterizations of the forward-model to be optimized using real experimental data collected from patients suffering from AF, as to better represent the inverse operator with respect to reconstruction metrics applicable to electrophysiology. The accuracy of our model is evaluated against a dataset of real-patient recordings to demonstrate its validity, and mathematical analysis is provided to support the polynomial expansion used in our inversion model.

show abstract

“…The sum of these two is given by E((f − P (x)) 2 ), where the expectation is with respect to the marginal distribution of x and the target function f is the conditional expectation of y given x. If the marginal distribution of x is known, then the split between approximation and sampling errors is no longer necessary, and one can obtain estimates as well as constructions directly from characteristics of the training data (e.g., [11,14,18]). In the classical paradigm where this distribution is not known, the approximation error decreases as the number of parameters in P increases to ∞.…”

Section: Introductionmentioning

confidence: 99%

An analysis of training and generalization errors in shallow and deep networks

Mhaskar

Poggio

2020

Neural Networks

View full text Add to dashboard Cite

This paper is motivated by an open problem around deep networks, namely, the apparent absence of overfitting despite large over-parametrization which allows perfect fitting of the training data. In this paper, we analyze this phenomenon in the case of regression problems when each unit evaluates a periodic activation function. We argue that the minimal expected value of the square loss is inappropriate to measure the generalization error in approximation of compositional functions in order to take full advantage of the compositional structure. Instead, we measure the generalization error in the sense of maximum loss, and sometimes, as a pointwise error. We give estimates on exactly how many parameters ensure both zero training error as well as a good generalization error. We prove that a solution of a regularization problem is guaranteed to yield a good training error as well as a good generalization error and estimate how much error to expect at which test data.

show abstract

Applications of classical approximation theory to periodic basis function networks and computational harmonic analysis

Cited by 6 publications

References 62 publications

Minimax estimation of the L<inf>1</inf> distance

Minimax estimation of the L<inf>1</inf> distance

Nonlinear electrocardiographic imaging using polynomial approximation networks

An analysis of training and generalization errors in shallow and deep networks

Contact Info

Product

Resources

About