Handbook of Differential Entropy

Michalowicz, Joseph V.; Nichols, Jonathan M.; Bucholtz, F.

doi:10.1201/b15991

Cited by 97 publications

(103 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…There is no closed form solution to Rician entropy [6], and when added to a Gaussian, the measurement distribution becomes even more complex. Using our method, we modeled first a zeroradius Rician, which is equivalent to a Rayleigh.…”

Section: A Radar Rangementioning

confidence: 99%

“…However, even for simple scenarios such as independent additive white Gaussian noise (AWGN), the resulting distributions can be complicated. Except for Gaussian source distributions, there are few closed form entropy expressions for independent additive random variables [6]. In addition, simulation or histogram methods can suffer from large biases [7].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Estimation information bounds using the I-MMSE formula and Gaussian mixture models

Paul

Bliss

2016

2016 Annual Conference on Information Science and Systems (CISS)

View full text Add to dashboard Cite

We derive a method to bound the mutual information between a noisy and noiseless measurement exploiting the I-MMSE estimation and information theory connection. Modeling the source distribution as a Gaussian mixture model, a closed form expression for upper and lower bounds of the minimum mean square error is found using recent results. Using the connection between rate of information relative to SNR and the minimum mean square error of the estimator, the mutual information can be bounded as well for arbitrary source distributions in Gaussian noise.

show abstract

Section: A Radar Rangementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Estimation information bounds using the I-MMSE formula and Gaussian mixture models

Paul

Bliss

2016

2016 Annual Conference on Information Science and Systems (CISS)

View full text Add to dashboard Cite

show abstract

“…The range, in general, is Rice distributed [10]. Unfortunately, there is no closed form solution for the differential entropy of a Rician [11]. However, we can compute this entropy using numerical integration or Monte Carlo methods [12].…”

Section: Measurement Model / Measurement Entropymentioning

confidence: 99%

Extending joint radar-communications bounds for FMCW radar with Doppler estimation

Paul

Bliss

2015

2015 IEEE Radar Conference (RadarCon)

View full text Add to dashboard Cite

We investigate cooperative in-band radar and communications signaling for frequency-modulated continuouswave (FMCW) radar and Doppler estimation. While each system typically considers the other system a source of interference, by considering the radar and communications operations to be a single system, joint performance bounds can be formulated. We extend previous work where a novel estimation and information theoretic bound formulation was constructed for a receiver that observes communications and radar returns in the same frequency allocation. While the previous work derived a joint performance bound in terms of the communications rate and the target delay estimation rate of the system for pulsed waveforms, we derive a similar bound for FMCW radar and include Doppler estimation. This extension is important given the rise of popularity of FMCW radars, and since the continuous signaling brings them closer to how communications systems operate. Comm TX Comm RXRadar TX/RX

show abstract

“…The application to the prediction of wind speed indicates that the variances of resampled time series increase almost exponentially with the increase of resampling period. Entropy in information theory is a powerful conception to describing the indeterminacy of a non-stationary time series [1,2] , and it has been used in a wide range of fields [3][4][5][6][7][8][9][10][11][12] . Its estimators from time series have also been studied extensively [13] .…”

mentioning

confidence: 99%

“…Entropy in information theory is a powerful conception to describing the indeterminacy of a non-stationary time series [1,2] , and it has been used in a wide range of fields [3][4][5][6][7][8][9][10][11][12] . Its estimators from time series have also been studied extensively [13] .…”

mentioning

confidence: 99%

Analytical fitting functions of finite sample discrete entropies of white Gaussian noise

Yang

Feng

Xiong

et al. 2015

Trans. Tianjin Univ.

View full text Add to dashboard Cite

： In order to find the convergence rate of finite sample discrete entropies of a white Gaussian noise(WGN), Brown entropy algorithm is numerically tested．With the increase of sample size, the curves of these finite sample discrete entropies are asymptotically close to their theoretical values．The confidence intervals of the sample Brown entropy are narrower than those of the sample discrete entropy calculated from its differential entropy, which is valid only in the case of a small sample size of WGN. The differences between sample Brown entropies and their theoretical values are fitted by two rational functions exactly, and the revised Brown entropies are more efficient. The application to the prediction of wind speed indicates that the variances of resampled time series increase almost exponentially with the increase of resampling period. Entropy in information theory is a powerful conception to describing the indeterminacy of a non-stationary time series [1,2] , and it has been used in a wide range of fields [3][4][5][6][7][8][9][10][11][12] . Its estimators from time series have also been studied extensively [13] . The differential entropy of a "true" white Gaussian noise(WGN), which follows a normal distribution and has an infinite sample size, is determined exactly by its variance of normal distribution [2] . But in practice, WGNs have finite sample sizes, often contain outliers and deviate from the normal distributions [14] . The exact and robust estimators of entropy and its variance from observed data are needed all the time [15] . In the prediction of non-stationary time series containing a WGN, the WGN forms an inaccessible limit of prediction accuracy. The identification of correct distribution function of the WGN from observed data is still an open problem [15] . In general, if the level of confidence keeps the same, the confidence intervals of a statistic get broader with the diminishing sample size. The smaller the sample size, the greater the differences between the estimated values of entropy or variance of a WGN and their true values [16] .Many studies show that the minimum error entropy (MEE)criterion can outperform the traditional mean square error criterion in supervised machine learning, especially in nonlinear and non-Gaussian situations [3] . Here the mean square error has the same calculating formula as the variance. Although Chen and Principe [3] explained this phenomenon by the smooth characteristic of entropy under the pollution due to an undesired random variable, the reason why the entropy is more reliable than the mean square error or the variance is still unknown. The content of this paper is organized as follows: for a pseudo-WGN with outliers and deviation from the normal distribution, the divergent intervals of its sample discrete entropies and its sample variances are calculated numerically; the convergence rates of sample discrete entropies which are asymptotical to their theoretical discrete entropies are fitted by two rational functions; the lost rate of signal in a wi...

show abstract

Handbook of Differential Entropy

Cited by 97 publications

References 0 publications

Estimation information bounds using the I-MMSE formula and Gaussian mixture models

Estimation information bounds using the I-MMSE formula and Gaussian mixture models

Extending joint radar-communications bounds for FMCW radar with Doppler estimation

Analytical fitting functions of finite sample discrete entropies of white Gaussian noise

Contact Info

Product

Resources

About