Efficient evaluation of the pdf of a random variable through the kernel density maximum entropy approach

Alibrandi, Umberto; Ricciardi, Giuseppe

doi:10.1002/nme.2300

Cited by 18 publications

(14 citation statements)

References 28 publications

(29 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Typically, a more efficient numerical procedure is used. By substituting

p_{i}^{ME} ((), λ)

into the definition of the Shannon's entropy (Equation ), it follows that the Lagrange multipliers are evaluated through the minimization of the following unconstrained convex functional:

Γ^{ME} ((),,, λ_{1} λ_{2} \dots λ_{M}) = λ_{0} ((),,, λ_{1} λ_{2} \dots λ_{M}) + \sum_{k = 1}^{M} λ_{k} G_{k},

where

λ_{0} ((),,, λ_{1} λ_{2} \dots λ_{M}) = normallog ({}, \sum_{i = 1}^{N} italicexp ((), - \sum_{k = 1}^{M} λ_{k} g_{k} ((), x_{i}))) .

…”

Section: Me Formalismmentioning

confidence: 99%

“…The main objective of this paper is to develop a unified framework based on the ME, applicable to random variables with any kind of support (bounded, semibounded, or unbounded) and able to provide the least biased reconstruction of the distribution including its tails, from the knowledge of a small number of available data. This is achieved by discretizing the moment problem representing the target PDF f X ( x ) as a sum of kernel densities f KDME ( x ; p ) ≅ f X ( x ) whose free parameters collected in the vector p are obtained by applying the ME principle to the discretized moment problem. Differently from Alibrandi and Ricciardi, here, the constraints of the ME method are the generalized moments, which include, as their subsets, the classical power and the fractional moments.…”

Section: Introductionmentioning

confidence: 99%

“…This is achieved by discretizing the moment problem representing the target PDF f X ( x ) as a sum of kernel densities f KDME ( x ; p ) ≅ f X ( x ) whose free parameters collected in the vector p are obtained by applying the ME principle to the discretized moment problem. Differently from Alibrandi and Ricciardi, here, the constraints of the ME method are the generalized moments, which include, as their subsets, the classical power and the fractional moments. The proposed approach is referred to as generalized kernel density maximum entropy method (GKDMEM) because it adopts a kernel density (KD) representation (KDR) of the target PDF, while the parameters are determined through the ME method, by using as constraints the generalized moments.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Kernel density maximum entropy method with generalized moments for evaluating probability distributions, including tails, from a small sample of data

Alibrandi

Mosalam

2017

Numerical Meth Engineering

View full text Add to dashboard Cite

In this paper, a novel method to determine the distribution of a random variable from a sample of data is presented. The approach is called generalized kernel density maximum entropy method, because it adopts a kernel density representation of the target distribution, while its free parameters are determined through the principle of maximum entropy (ME). Here, the ME solution is determined by assuming that the available information is represented from generalized moments, which include as their subsets the power and the fractional ones. The proposed method has several important features: (1) applicable to distributions with any kind of support, (2) computational efficiency because the ME solution is simply obtained as a set of systems of linear equations, (3) good tradeoff between bias and variance, and (4) good estimates of the tails of the distribution, in the presence of samples of small size. Moreover, the joint application of generalized kernel density maximum entropy with a bootstrap resampling allows to define credible bounds of the target distribution. The method is first benchmarked through an example of stochastic dynamic analysis. Subsequently, it is used to evaluate the seismic fragility functions of a reinforced concrete frame, from the knowledge of a small set of available ground motions. /journal/nme other quantities derived from the data. The first general representations for a PDF have adopted Hermite polynomials, eg, A-type and C-type Gram Charlier series and the Edgeworth series expansion. 3,4 An improvement is represented by the model proposed by Winterstein, 5 and it is largely applied in engineering.The choice of the distribution type is an open issue, from a conceptual and practical point of view. In the presence of limited information (sample of small size and/or lower-order moments), in the most general case, there is no theoretical justification to prefer one distribution over another. In such case, an attractive technique is based on the principle of maximum entropy (ME). [6][7][8] The ME distribution represents the least biased distribution given the available information. However, the application of the ME principle, as typically adopted in literature, has some practical and theoretical limitations.If the power moments represent the available information, then the ME PDF f ME (x) may require a large number M of moments (M ≥ 4) to accurately describe the tails of the distribution. However, the moment problem is an ill-posed problem. 9 Thus, for large M, the entropy maximization algorithm experiences numerical instability. Moreover, at the tails, the ME distribution oscillates because of the nonmonotonic nature of the polynomial embedded in the f ME (x). Thus, only the lower-order moments are typically considered, but in such case, f ME (x) hardly models tails fatter than the Gaussian. Therefore, the tails of many distributions cannot be well fitted by the ME distribution with M ≤ 4. 10 Furthermore, from a practical point of view, we do not have the true moments, but samples of data, f...

show abstract

“…Typically, a more efficient numerical procedure is used. By substituting

p_{i}^{ME} ((), λ)

into the definition of the Shannon's entropy (Equation ), it follows that the Lagrange multipliers are evaluated through the minimization of the following unconstrained convex functional:

Γ^{ME} ((),,, λ_{1} λ_{2} \dots λ_{M}) = λ_{0} ((),,, λ_{1} λ_{2} \dots λ_{M}) + \sum_{k = 1}^{M} λ_{k} G_{k},

where

λ_{0} ((),,, λ_{1} λ_{2} \dots λ_{M}) = normallog ({}, \sum_{i = 1}^{N} italicexp ((), - \sum_{k = 1}^{M} λ_{k} g_{k} ((), x_{i}))) .

…”

Section: Me Formalismmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Kernel density maximum entropy method with generalized moments for evaluating probability distributions, including tails, from a small sample of data

Alibrandi

Mosalam

2017

Numerical Meth Engineering

View full text Add to dashboard Cite

show abstract

“…In formula (9), negentropy is denoted by pdf of signals. Kernel density maximum entropy approach (KD-MEM) is used to estimate the pdf of signals because of its excellent performance using very few observation values [5] .…”

Section: Adaptive Echo Cancellation Based On Bssmentioning

confidence: 99%

An Adaptive Echo Cancellation Method Based on a Blind Signal Separation

Zhengquan

2010

2010 International Conference on Electrical and Control Engineering

View full text Add to dashboard Cite

This paper presents a new adaptive echo cancellation (AEC) method base on a blind signal separation (BSS) to reduce the impact of the background noise and echo of the enclosure room on the speech communication signal. The correlation and negentropy of signals are used to distinguish the useful speech signal from several separated signals to solve the problem of the uncertainty of BSS. During single talk, this method separates the echo from noise using the echo mixture as a reference signal to AEC, by which the performance of AEC is improved. During double talk, the near-end speech signal can be accurately separated from the mixture signals by BSS. The experiment resutlts show that our algorithm can raise the output SNR of the canceler by about 10 dB compaired to NLMS algorithm .

show abstract

“…Alibrandi and Ricciardi (2008) reported data on 57 wind speed measurements made at each of five different elevations in Italy's Messina Strait region. The dataset array windspeed, stored in windspeed.mat, contains these data.…”

Section: Univariate Example: Wind Speed Datamentioning

confidence: 99%

A Greedy Algorithm for Unimodal Kernel Density Estimation by Data Sharpening

Wolters¹

2012

J. Stat. Soft.

View full text Add to dashboard Cite

We consider the problem of nonparametric density estimation where estimates are constrained to be unimodal. Though several methods have been proposed to achieve this end, each of them has its own drawbacks and none of them have readily-available computer codes. The approach of Braun and Hall (2001), where a kernel density estimator is modified by data sharpening, is one of the most promising options, but optimization difficulties make it hard to use in practice. This paper presents a new algorithm and MATLAB code for finding good unimodal density estimates under the Braun and Hall scheme. The algorithm uses a greedy, feasibility-preserving strategy to ensure that it always returns a unimodal solution. Compared to the incumbent method of optimization, the greedy method is easier to use, runs faster, and produces solutions of comparable quality. It can also be extended to the bivariate case.

show abstract

Efficient evaluation of the pdf of a random variable through the kernel density maximum entropy approach

Cited by 18 publications

References 28 publications

Kernel density maximum entropy method with generalized moments for evaluating probability distributions, including tails, from a small sample of data

Kernel density maximum entropy method with generalized moments for evaluating probability distributions, including tails, from a small sample of data

An Adaptive Echo Cancellation Method Based on a Blind Signal Separation

A Greedy Algorithm for Unimodal Kernel Density Estimation by Data Sharpening

Contact Info

Product

Resources

About