Goodness-of-Fit Testing for Hölder-Continuous Densities: Sharp Local Minimax Rates

Chhor, Julien; Carpentier, Alexandra

doi:10.48550/arxiv.2109.04346

Cited by 3 publications

(3 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The choice of t influences the minimax separation distance. The effect of the L t separation distance for t 2 OE1; 2 has also been investigated in the paper [22] in the case of goodness-of-fit testing for Hölder continuous densities.…”

Section: Influence Of the `T Normmentioning

confidence: 99%

Sharp local minimax rates for goodness-of-fit testing in multivariate binomial and Poisson families and in multinomials

Chhor

Carpentier

2022

Math. Stat. Learn.

View full text Add to dashboard Cite

We consider the identity testing problem -or goodness-of-fit testing problemin multivariate binomial families, multivariate Poisson families and multinomial distributions. Given a known distribution p and n i.i.d. samples drawn from an unknown distribution q, we investigate how large > 0 should be to distinguish, with high probability, the case p D q from the case d.p; q/, where d denotes a specific distance over probability distributions. We answer this question in the case of a family of different distances: d.p; q/ D kp qk t for t 2 OE1; 2, where k k t is the entrywise `t norm. Besides being locally minimax-optimal -i.e. characterizing the detection threshold in dependence of the known matrix p -our tests have simple expressions and are easily implementable.

show abstract

Section: Influence Of the `T Normmentioning

confidence: 99%

Sharp local minimax rates for goodness-of-fit testing in multivariate binomial and Poisson families and in multinomials

Chhor

Carpentier

2022

Math. Stat. Learn.

View full text Add to dashboard Cite

show abstract

“…Butucea and Tribouley (2006) study two-sample testing in more general Besov spaces, but for onedimensional densities; they also prove adaptivity. See also Balakrishnan and Wasserman (2018); Donoho and Jin (2015); Jin and Ke (2016); Chhor and Carpentier (2021); Dubois et al (2021); for reviews and further related works.…”

Section: Related Workmentioning

confidence: 99%

T-Cal: An optimal test for the calibration of predictive models

Lee¹,

Huang²,

Hassani³

et al. 2022

Preprint

View full text Add to dashboard Cite

The prediction accuracy of machine learning methods is steadily increasing, but the calibration of their uncertainty predictions poses a significant challenge. Numerous works focus on obtaining well-calibrated predictive models, but less is known about reliably assessing model calibration. This limits our ability to know when algorithms for improving calibration have a real effect, and when their improvements are merely artifacts due to random noise in finite datasets. In this work, we consider detecting miscalibration of predictive models using a finite validation dataset as a hypothesis testing problem. The null hypothesis is that the predictive model is calibrated, while the alternative hypothesis is that the deviation from calibration is sufficiently large.We find that detecting mis-calibration is only possible when the conditional probabilities of the classes are sufficiently smooth functions of the predictions. When the conditional class probabilities are Hölder continuous, we propose T-Cal, a minimax optimal test for calibration based on a debiased plug-in estimator of the 2-Expected Calibration Error (ECE). We further propose Adaptive T-Cal, a version that is adaptive to unknown smoothness. We verify our theoretical findings with a broad range of experiments, including with several popular deep neural net architectures and several standard post-hoc calibration methods. T-Cal is a practical general-purpose tool, which-combined with classical tests for discrete-valued predictors-can be used to test the calibration of virtually any probabilistic classification method. T-Cal is available at https://github.com/dh7401/T-Cal.

show abstract

“…The paper [Wag15] also highlighted this duality in the global case rather than in the local one. The paper [CC21] considered an analogous version of Problem (40), for Hölder-continuous densities.…”

Section: Multinomial Testingmentioning

confidence: 99%

Sparse Signal Detection in Heteroscedastic Gaussian Sequence Models: Sharp Minimax Rates

Chhor¹,

Mukherjee²,

Sen³

2022

Preprint

View full text Add to dashboard Cite

Given a heterogeneous Gaussian sequence model with unknown mean θ ∈ R d and known covariance matrix Σ = diag(σ 2 1 , . . . , σ 2 d ), we study the signal detection problem against sparse alternatives, for known sparsity s. Namely, we characterize how large ǫ * > 0 should be, in order to distinguish with high probability the null hypothesis θ = 0 from the alternative composed of s-sparse vectors in R d , separated from 0 in L t norm (t ≥ 1) by at least ǫ * . We find minimax upper and lower bounds over the minimax separation radius ǫ * and prove that they are always matching. We also derive the corresponding minimax tests achieving these bounds. Our results reveal new phase transitions regarding the behavior of ǫ * with respect to the level of sparsity, to the L t metric, and to the heteroscedasticity profile of Σ. In the case of the Euclidean (i.e. L 2 ) separation, we bridge the remaining gaps in the literature.

show abstract

Goodness-of-Fit Testing for Hölder-Continuous Densities: Sharp Local Minimax Rates

Cited by 3 publications

References 20 publications

Sharp local minimax rates for goodness-of-fit testing in multivariate binomial and Poisson families and in multinomials

Sharp local minimax rates for goodness-of-fit testing in multivariate binomial and Poisson families and in multinomials

T-Cal: An optimal test for the calibration of predictive models

Sparse Signal Detection in Heteroscedastic Gaussian Sequence Models: Sharp Minimax Rates

Contact Info

Product

Resources

About