Random Forest Weighted Local Fréchet Regression

Abstract

Statistical analysis is increasingly confronted with complex data from general metric spaces, such as symmetric positive definite matrix-valued data and probability distribution functions. [47] and [17] establish a general paradigm of Fréchet regression with complex metric space valued responses and Euclidean predictors. However, their proposed local Fréchet regression approach involves nonparametric kernel smoothing and suffers from the curse of dimensionality. To address this issue, we in this paper propose a novel random forests weighted local Fréchet regression paradigm. The main mechanism of our approach relies on the adaptive kernels generated by random forests. Our first method utilizes these weights as the local average to solve the Fréchet mean, while the second method performs local linear Fréchet regression, making both methods locally adaptive. Our proposals significantly improve existing Fréchet regression methods. Based on the theory of infinite order U-processes and infinite order Mm n -estimator, we establish the consistency, rate of convergence, and asymptotic normality for our proposed random forests weighted Fréchet regression estimator, which covers the current large sample theory of random forests with Euclidean responses as a special case. Numerical studies show the superiority of our proposed two methods for Fréchet regression with several commonly encountered types of responses such as probability distribution functions, symmetric positive definite matrices, and sphere data. The practical merits of our proposals are also demonstrated through the application to the human mortality distribution data.

SPD-DDPM: Denoising Diffusion Probabilistic Models in the Symmetric Positive Definite Space

“…Previous methods often use traditional statistical modeling frameworks as discriminative models by estimating E(X|y) given the conditional variables y. For example, they define an additive model in a reproducing kernel Hilbert space using kernel functions (Lin, Müller, and Park 2023) or define a regression model in a metric space upon Fréchet mean (Petersen and Müller 2019;Qiu, Yu, and Zhu 2022). However, these models often face challenges when dealing with high-dimensional SPD matrices or predictor y.…”

“…Based on this, Tucker et al(Tucker, Wu, and Müller 2023) proposed a method for variable selection. Lin et al(Lin, Müller, and Park 2023) proposed an adaptive model in SPD space Qiu et al (Qiu, Yu, and Zhu 2022). propose random forest with SPD matrix response.…”