On the Shannon theory of information transmission in the case of continuous signals

Kolmogorov, A. N.

doi:10.1109/tit.1956.1056823

Cited by 348 publications

(253 citation statements)

References 0 publications

Supporting

Mentioning

242

Contrasting

Unclassified

Order By: Relevance

“…Their mutual information, M (x, y) = p(x, y) log[p(x, y)/p(x)p(y)]dxdy, can be defined as the upper bound of the mutual information between all discretizations of x and y (Kolmogorov, 1956). Behind this definition lies the crucial fact that when refining the partitions of the sample space used to discretize x and y, the discrete mutual information must increase.…”

Section: B2 a New Information Measure For Continuous Random Variablesmentioning

confidence: 99%

Kernel independent component analysis

Bach

Jordan

2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03).

553

992

View full text Add to dashboard Cite

We present a class of algorithms for Independent Component Analysis (ICA) which use contrast functions based on canonical correlations in a reproducing kernel Hilbert space. On the one hand, we show that our contrast functions are related to mutual information and have desirable mathematical properties as measures of statistical dependence. On the other hand, building on recent developments in kernel methods, we show that these criteria and their derivatives can be computed efficiently. Minimizing these criteria leads to flexible and robust algorithms for ICA. We illustrate with simulations involving a wide variety of source distributions, showing that our algorithms outperform many of the presently known algorithms.

show abstract

Section: B2 a New Information Measure For Continuous Random Variablesmentioning

confidence: 99%

Kernel independent component analysis

Bach

Jordan

2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03).

553

992

View full text Add to dashboard Cite

show abstract

“…Particularly in communications systems, the Fourier series and Fourier transform [13] are tools to generate or recover signals from cardinal series. The aliasing phenomenon in Shannon sampling reconstruction procedure has been discussed in numerous papers [14][15][16]. The presence of aliasing in the generation of signals often is associated to truncated cardinal series [14].…”

Section: Aliasingmentioning

confidence: 99%

Undesired Changes in PID Controller Model Due to Simulators

Paz-Peña¹,

Vargas-Jarillo²

2013

Proceedings of the 2nd International Symposium on Computer, Communication, Control and Automation

View full text Add to dashboard Cite

Abstract-In this work we propose the simulation of a PID controller to solve two problems, namely: 1) Recovery signal with perturbation, using Simulink. 2) Tracking signal for a motor with perturbation in two ways, using Simulink and using a Matlab program. The aim of this work is to show particular examples, where Simulink changes the expected behavior of the model. We emphasize the importance of numerical integration of differential equations and the bounded signals treatment

show abstract

“…where on the right side of (4) is the directed information between two sequences of length n defined in (1); and in (5) t 0 = 0 by convention and the mutual information terms between two continuous time processes, conditioned on a third, are well-defined objects, as developed in [16], [17]. The mutual information of two random variables U, V with continuous alphabets, i.e., I(U ; V ) is defined as…”

Section: Definition Of Directed Informationmentioning

confidence: 99%

“…e R → 0 as P (T ) e → 0, the equality in (32) follows from the fact that X ti−1+∆ is a deterministic function of M and Y ti−1 0 , the equality in (33) follows from the assumption that t i − t i−1 < ∆, the equality in (35) follows from (14), and the equality in (36) follows from (16). Hence, we obtained that for every t…”

Section: T T=0mentioning

confidence: 99%