Adaptive nonparametric regression with the K-nearest neighbour fused lasso

Padilla, Oscar Hernán Madrid; Sharpnack, James; Chen, Yanzhen; Witten, Daniela

doi:10.1093/biomet/asz071

Cited by 12 publications

(9 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fruitful avenues for future work include extending the framework to tensor settings (Sun and Li, 2019), and exploring other optimization frameworks such as semi-smooth Newton (Yuan et al, 2018;Sun et al, 2018) and stochastic descent algorithms (Panahi et al, 2017) that may lead to further computational gains. Similar penalties involving k-NN affinities within a fusion penalty have recently been studied for non-parametric regression (Madrid Padilla et al, 2020). Similar to their benefits in our setting, the k-NN terms enable "manifold adaptivity", in addition to the fusion term which provides local adaptivity.…”

Section: Discussionmentioning

confidence: 79%

Biconvex Clustering

Chakraborty,

2020

Preprint

View full text Add to dashboard Cite

Convex clustering has recently garnered increasing interest due to its attractive theoretical and computational properties. While it confers many advantages over traditional clustering methods, its merits become limited in the face of high-dimensional data. In such settings, not only do Euclidean measures of fit appearing in the objective provide weaker discriminating power, but pairwise affinity terms that rely on k-nearest neighbors become poorly specified. We find that recent attempts which successfully address the former difficulty still suffer from the latter, in addition to incurring high computational cost and some numerical instability. To surmount these issues, we propose to modify the convex clustering objective so that feature weights are optimized jointly with the centroids. The resulting problem becomes biconvex and as such remains well-behaved statistically and algorithmically. In particular, we derive a fast algorithm with closed form updates and convergence guarantees, and establish finite-sample bounds on its prediction error that imply consistency. Our biconvex clustering method performs feature selection throughout the clustering task: as the learned weights change the effective feature representation, pairwise affinities can be updated adaptively across iterations rather than precomputed within a dubious feature space. We validate the contributions on real and simulated data, showing that our method effectively addresses the challenges of dimensionality while reducing dependence on carefully tuned heuristics typical of existing approaches.

show abstract

Section: Discussionmentioning

confidence: 79%

Biconvex Clustering

Chakraborty,

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…(5) is essentially nonparametric (Tibshirani et al 2005;Madrid Padilla et al 2020), and θ * u can be viewed as the value that some function f 0 (•) takes on device u, in which case a device is a data point. In particular, if further p = 1 and G is a grid graph, ( 5) is reduced to total variation de-noising (Hütter and Rigollet 2016).…”

Section: Methodsmentioning

confidence: 99%

Heterogeneous Federated Learning on a Graph

Wang¹,

Zhao²,

Lin³

2022

Preprint

View full text Add to dashboard Cite

Federated learning, where algorithms are trained across multiple decentralized devices without sharing local data, is increasingly popular in distributed machine learning practice. Typically, a graph structure G exists behind local devices for communication.In this work, we consider parameter estimation in federated learning with data distribution and communication heterogeneity, as well as limited computational capacity of local devices. We encode the distribution heterogeneity by parametrizing distributions on local devices with a set of distinct p-dimensional vectors. We then propose to jointly estimate parameters of all devices under the M -estimation framework with the fused Lasso regularization, encouraging an equal estimate of parameters on connected devices in G. We provide a general result for our estimator depending on G, which can be further calibrated to obtain convergence rates for various specific problem setups. Surprisingly, our estimator attains the optimal rate under certain graph fidelity condition on G, as if we could aggregate all samples sharing the same distribution. If the graph fidelity condition is not met, we propose an edge selection procedure via multiple testing to ensure the optimality. To ease the burden of local computation, a decentralized stochastic version of ADMM is provided, with convergence rate O(T −1 log T ) where T denotes the number of iterations. We highlight that, our algorithm transmits only parameters along edges of G at each iteration, without requiring a central machine, which preserves privacy. To address communication heterogeneity, we further extend it to the case where devices are randomly inaccessible during the training process, with a similar algorithmic convergence guarantee. The computational and statistical efficiency of our method is evidenced by simulation experiments and the 2020 US presidential election data set.

show abstract

“…Instead, we can determine it by a data-driven information criterion approach described later in this section. Finally, besides its capability to capture piece-wise constant coefficients, previous theoretical studies proved that this penalty has a strong local adaptivity in that it is also capable of capturing piecewise Lipschitz continuous functions [35], which implies that the method can also approximate a spatially smoothly varying contamination probability reasonably well.…”

Section: Estimation Of Spatially Clustered Contamination Probabilitiesmentioning

confidence: 98%

Risk Based Arsenic Rational Sampling Design for Public and Environmental Health Management

Yin¹,

Sang²,

Schnoebelen³

et al. 2021

Preprint

View full text Add to dashboard Cite

Groundwater contaminated with arsenic has been recognized as a global threat, which negatively impacts human health. Populations that rely on private wells for their drinking water are vulnerable to the potential arsenic-related health risks such as cancer and birth defects. Arsenic exposure through drinking water is among one of the primary arsenic exposure routes that can be effectively managed by active testing and water treatment. From the public and environmental health management perspective, it is critical to allocate the limited resources to establish an effective arsenic sampling and testing plan for health risk mitigation. We present a spatially adaptive sampling design approach based on an estimation of the spatially varying underlying contamination distribution. The method is different from traditional sampling design methods that often rely on a spatially constant or smoothly varying contamination distribution. In contrast, we propose a statistical regularization method to automatically detect spatial clusters of the underlying contamination risk from the currently available private well arsenic testing data in the USA, Iowa. This approach allows us to develop a sampling design method that is adaptive to the changes in the contamination risk across the identified clusters. We provide the spatially adaptive sample size calculation and sampling location determination at different acceptance precision and confidence levels for each cluster. The spatially adaptive sampling approach may effectively mitigate the arsenic risk from the resource management perspectives. The model presents a framework that can be widely used for other environmental contaminant monitoring and sampling for public and environmental health.

show abstract

Adaptive nonparametric regression with the K-nearest neighbour fused lasso

Cited by 12 publications

References 31 publications

Biconvex Clustering

Biconvex Clustering

Heterogeneous Federated Learning on a Graph

Risk Based Arsenic Rational Sampling Design for Public and Environmental Health Management

Contact Info

Product

Resources

About