Inverse transformed density rejection for unbounded monotone densities

Hörmann, Wolfgang; Leydold, Josef; Derflinger, Gerhard

doi:10.1145/1276927.1276931

Cited by 4 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 1 summarizes the main techniques proposed in literature for this purpose. In some specific situations, rejection samplers [5,22,23,33,49] and their adaptive version, as the adaptive rejection sampler (ARS) [16], are employed to generate one sample from each πd per iteration. Since the standard ARS can be applied only to log-concave densities, several extensions have been introduced [21,17,37].…”

Section: Monte Carlo-within-gibbs Sampling Schemesmentioning

confidence: 99%

“…The ARS algorithms are very appealing techniques since they construct a non-parametric proposal to mimic the shape of the target pdf, yielding in general excellent performance (i.e., independent samples from πd with a high acceptance rate). samplers (Ca↵o et al, 2002;Hörmann, 2002;Hörmann et al, 2007;Marrelec and Benali, 2004;Tanizaki, 1999) and their adaptive version, as the adaptive rejection sampler (ARS) Gilks and Wild (1992), are employed to generate one sample from each ⇡d per iteration. Since the standard ARS can be applied only to lo-concave densities, several extensions have been introduced Hörmann (1995a); Görür and Teh (2011); Martino and Míguez (2011).…”

Section: Monte Carlo-within-gibbs Sampling Schemesmentioning

confidence: 99%

See 1 more Smart Citation

The Recycling Gibbs sampler for efficient learning

Martino

Elvira

Camps‐Valls

2018

Digital Signal Processing

View full text Add to dashboard Cite

Monte Carlo methods are essential tools for Bayesian inference. Gibbs sampling is a well-known Markov chain Monte Carlo (MCMC) algorithm, extensively used in signal processing, machine learning, and statistics, employed to draw samples from complicated high-dimensional posterior distributions. The key point for the successful application of the Gibbs sampler is the ability to draw efficiently samples from the full-conditional probability density functions. Since in the general case this is not possible, in order to speed up the convergence of the chain, it is required to generate auxiliary samples whose information is eventually disregarded. In this work, we show that these auxiliary samples can be recycled within the Gibbs estimators, improving their efficiency with no extra cost. This novel scheme arises naturally after pointing out the relationship between the standard Gibbs sampler and the chain rule used for sampling purposes. Numerical simulations involving simple and real inference problems confirm the excellent performance of the proposed scheme in terms of accuracy and computational efficiency. In particular we give empirical evidence of performance in a toy example, inference of Gaussian processes hyperparameters, and learning dependence graphs through regression.

show abstract

Section: Monte Carlo-within-gibbs Sampling Schemesmentioning

confidence: 99%

Section: Monte Carlo-within-gibbs Sampling Schemesmentioning

confidence: 99%

The Recycling Gibbs sampler for efficient learning

Martino

Elvira

Camps‐Valls

2018

Digital Signal Processing

View full text Add to dashboard Cite

show abstract

“…However, even sampling from π can often be complicated. In some specific situations, rejection samplers [41][42][43][44][45] and their adaptive versions, adaptive rejection sampling (ARS) algorithms, are employed to generate (one) sample from π [12, 19, 25, 27-29, 40, 46, 47]. The ARS algorithms are very appealing techniques since they construct a non-parametric proposal in order to mimic the shape of the target pdf, yielding in general excellent performance (i.e., independent samples from π with an high acceptance rate).…”

Section: Range Of Applicability and Multivariate Generationmentioning

confidence: 99%

Adaptive independent sticky MCMC algorithms

Martino

Casarin

Leisen

et al. 2018

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

Monte Carlo methods have become essential tools to solve complex Bayesian inference problems in different fields, such as computational statistics, machine learning, and statistical signal processing. In this work, we introduce a novel class of adaptive Monte Carlo methods, called adaptive independent sticky Markov Chain Monte Carlo (MCMC) algorithms, to sample efficiently from any bounded target probability density function (pdf). The new class of algorithms employs adaptive non-parametric proposal densities, which become closer and closer to the target as the number of iterations increases. The proposal pdf is built using interpolation procedures based on a set of support points which is constructed iteratively from previously drawn samples. The algorithm's efficiency is ensured by a test that supervises the evolution of the set of support points. This extra stage controls the computational cost and the convergence of the proposal density to the target. Each part of the novel family of algorithms is discussed and several examples of specific methods are provided. Although the novel algorithms are presented for univariate target densities, we show how they can be easily extended to the multivariate context by embedding them within a Gibbs-type sampler or the hit and run algorithm. The ergodicity is ensured and discussed. An overview of the related works in the literature is also provided, emphasizing that several well-known existing methods (like the adaptive rejection Metropolis sampling (ARMS) scheme) are encompassed by the new class of algorithms proposed here. Eight numerical examples (including the inference of the hyper-parameters of Gaussian processes, widely used in machine learning for signal processing applications) illustrate the efficiency of sticky schemes, both as stand-alone methods to sample from complicated one-dimensional pdfs and within Gibbs samplers in order to draw from multi-dimensional target distributions.

show abstract