Estimation of the mean function of functional data via deep neural networks

Abstract

In this work, we propose a deep neural networks-based method to perform nonparametric regression for functional data. The proposed estimators are based on sparsely connected deep neural networks with rectifier linear unit (ReLU) activation function. We provide the convergence rate of the proposed deep neural networks estimator in terms of the empirical norm. Through Monte Carlo simulation studies, we examine the finite sample performance of the proposed method. Finally, the proposed method is applied to analyse positron emission tomography images of patients with Alzheimer's disease obtained from the Alzheimer Disease Neuroimaging Initiative database.

Robust Deep Neural Network Estimation for Multi-dimensional Functional Data

“…It is interesting to observe that our proposed RDNN estimator enjoys the same asymptotic as the least squares DNN estimator [24] does. Specifically, the convergence rate for M-type DNN estimator in the functional regression model depends on the smoothness, i.e., β * , and the intrinsic dimension, i.e., t , of the true mean function f 0 , and the decay rate of the maximal value of the variance function E{(ψ( 1j )) 2 }.…”

“…Different from the classical SGD procedures, Adam is a method for efficient stochastic optimization that only requires firstorder gradients with little memory requirement. Hence, it is well suited for problems when there are large sample sizes and parameters ( [11]), and is widely used in network training for FDA, such as [24]. In our numerical studies, Adam provides the best results and is the most computationally efficient among other gradient based algorithms.…”

“…For the first part of Assumption (A6), when considering the classical L 2 loss, it essentially makes sure the maximal value of the covariance function is finite and decreases when the number of measurements increases. They are standard regularity conditions for the covariance functions in FDA literature, see [4,14,24] for example. The second part of Assumption (A6) essentially requires that for any j = 1, .…”

“…There are also some works proposed for deep learning algorithms for FDA from the statistical point of view [22]. Based on the sparsely connected DNN, [24] proposed a DNN estimator for the mean function from functional data based on the least squares neural network regression. However, none of them works on the robust statistics, not to mention the proven theoretical results for robust FDA.…”

Deep Feature Screening: Feature Selection for Ultra High-Dimensional Data via Deep Neural Networks

“…Farrell et al (2021) studies the rates of convergence for deep feedforward neural nets in semiparametric inference. The successful applications include, but are not limited to, computer vision (He et al, 2016), natural language processing (Bahdanau et al, 2014), drug discovery and toxicology (Jiménez-Luna et al, 2020), and dynamics system (Li et al, 2021), functional data analysis (Wang et al, 2021).…”