Product-form estimators: exploiting independence to scale up Monte Carlo

Abstract

We introduce a class of Monte Carlo estimators that aim to overcome the rapid growth of variance with dimension often observed for standard estimators by exploiting the target’s independence structure. We identify the most basic incarnations of these estimators with a class of generalized U-statistics and thus establish their unbiasedness, consistency, and asymptotic normality. Moreover, we show that they obtain the minimum possible variance amongst a broad class of estimators, and we investigate their computational cost and delineate the settings in which they are most efficient. We exemplify the merger of these estimators with other well known Monte Carlo estimators so as to better adapt the latter to the target’s independence structure and improve their performance. We do this via three simple mergers: one with importance sampling, another with importance sampling squared, and a final one with pseudo-marginal Metropolis–Hastings. In all cases, we show that the resulting estimators are well founded and achieve lower variances than their standard counterparts. Lastly, we illustrate the various variance reductions through several examples.

A Divide and Conquer Sequential Monte Carlo Approach to High Dimensional Filtering

“…If u is a leaf node, the algorithm performs a simple importance sampling step with proposal K u and importance weight w u := γ u /K u to obtain a weighted particle population {x n u , w n u } N n=1 approximating γ u . Otherwise, to obtain a particle population approximating γ u , we gather the particle product form estimator (Kuntz et al, 2022)…”

Asymptotic distributions of a new type of design‐based incomplete U‐statistics

“…ICUDO has been discussed in some recent research regarding efficient incomplete U‐statistics and Monte Carlo methods. We refer to Chan et al (2021), Huang et al (2022), Kuntz et al (2021), Kuntz et al (2022), Maurer (2022) and Zhang and Xia (2022) for more details. In Maurer (2022), the author finds “Kong and Zheng (2020) propose a sophisticated design strategy, which also takes the values of the into account.…”

A principled stopping rule for importance sampling

“…Often, μ * h is the unnormalized IS (UIS) estimator or the self-normalized IS (SNIS) estimator. Other IS estimators have also been proposed (see Elvira et al, 2019, Kuntz et al, 2022, Martino et al, 2018, Vehtari et al, 2015.…”