An Introduction to Information Theory.

Mallows, C. L.; Reza, F.M.

doi:10.2307/2311344

Cited by 203 publications

(21 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…At the heart of information theory lies the mutual information, Shannon's fundamental theorem ( [17], Chapter 12) states that the mutual information corresponding to a measurement is the average amount of error-free data which may be gained through the measurement of system. Information is a natural tool and concept in communications and physics, in an operation of measurement or communication, one may seek to maximize the gained information.…”

Section: Entropic Uncertainty Relation Proved By Maassen Andmentioning

confidence: 99%

Holevo bound of entropic uncertainty in Schwarzschild spacetime

et al. 2018

View full text Add to dashboard Cite

For a pair of incompatible quantum measurements, the total uncertainty can be bounded by a state-independent constant. However, such a bound can be violated if the quantum system is entangled with another quantum system (called memory); the quantum correlation between the systems can reduce the measurement uncertainty. On the other hand, in a curved spacetime, the presence of the Hawking radiation can increase the uncertainty in quantum measurement. The interplay of quantum correlation in the curved spacetime has become an interesting arena for studying quantum uncertainty relations. Here we demonstrate that the bounds of the entropic uncertainty relations, in the presence of memory, can be formulated in terms of the Holevo quantity, which limits how much information can be encoded in a quantum system. Specifically, we considered two examples with Dirac fields, near the event horizon of a Schwarzschild black hole, the Holevo bound provides a better bound than the previous bound based on the mutual information. Furthermore, if the memory moves away from the black hole, the difference between the total uncertainty and the Holevo bound remains a constant, not depending on any property of the black hole.

show abstract

Section: Entropic Uncertainty Relation Proved By Maassen Andmentioning

confidence: 99%

Holevo bound of entropic uncertainty in Schwarzschild spacetime

et al. 2018

View full text Add to dashboard Cite

show abstract

“…where 𝐼𝐼(𝑈𝑈, 𝑉𝑉) is the mutual information of 𝑈𝑈 and 𝑉𝑉, and 𝐻𝐻(𝑈𝑈) and 𝐻𝐻(𝑉𝑉) are the entropies of 𝑈𝑈 and 𝑉𝑉 respectively 58 . We utilize the functions ARI and AMI in R package aricode to calculate ARI and AMI, respectively.…”

Section: Regularization In Autoencodermentioning

confidence: 99%

Exploring the Optimization of Autoencoder Design for Imputing Single-Cell RNA Sequencing Data

2023

Preprint

View full text Add to dashboard Cite

Autoencoders are the backbones of many imputation methods that aim to relieve the sparsity issue in single-cell RNA sequencing (scRNA-seq) data. The imputation performance of an autoencoder relies on both the neural network architecture and the hyperparameter choice. So far, literature in the single-cell field lacks a formal discussion on how to design the neural network and choose the hyperparameters. Here, we conducted an empirical study to answer this question. Our study used many real and simulated scRNA-seq datasets to examine the impacts of the neural network architecture, the activation function, and the regularization strategy on imputation accuracy and downstream analyses. Our results show that (i) deeper and narrower autoencoders generally lead to better imputation performance; (ii) the sigmoid and tanh activation functions consistently outperform other commonly used functions including ReLU; (iii) regularization improves the accuracy of imputation and downstream cell clustering and DE gene analyses. Notably, our results differ from common practices in the computer vision field regarding the activation function and the regularization strategy. Overall, our study offers practical guidance on how to optimize the autoencoder design for scRNA-seq data imputation.

show abstract

“…A basic communication model usually consists of three indispensable parts: the senders (or sources), the receivers (or sinks), and the communication channel that carries information from the senders to the receivers [20]. The communication channel determines the basic nature of the model and usually is the most important of these components [21].…”

Section: B3 Parallel Communication Modelmentioning

confidence: 99%

“…where t  indicates the time difference between two sequential sampling time points of the signal [20]. Finally, the linear operator A can be estimated as follows:…”

Section: B Eeg Subsystem Decomposition Based On Dmdmentioning

confidence: 99%

Assessment of Multivariate Information Transmission in Space-Time-Frequency Domain: A Case Study for EEG Signals

Han

et al. 2023

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

Objective: Multivariate signal (MS) analysis, especially the assessment of its information transmission (for example, from the perspective of network science), is the key to our understanding of various phenomena in biology, physics and economics. Although there is a large amount of literature demonstrating that MS can be decomposed into space-timefrequency domain information, there seems to be no research confirming that multivariate information transmission (MIT) in these three domains can be quantified. Therefore, in this study, we attempted to combine dynamic mode decomposition (DMD) and parallel communication model (PCM) together to realize it. Methods: We first regarded MS as a large-scale system and then used DMD to decompose it into specific subsystems with their own intrinsic oscillatory frequencies. At the same time, the transition probability matrix (TPM) of information transmission within and between MS at two consecutive moments in each subsystem can also be calculated. Then, communication parameters (CPs) derived from each TPM were calculated in order to quantify the MIT in the space-time-frequency domain. In this study, multidimensional electroencephalogram (EEG) signals were used to illustrate our method. Results: Compared with traditional EEG brain networks, this method shows greater potential in EEG analysis to distinguish between patients and healthy controls. Conclusion:This study demonstrates the feasibility of measuring MIT in the space-time-frequency domain simultaneously. Significance: This study shows that MIT analysis in the spacetime-frequency domain is not only completely different from the MS decomposition in these three domains, but also can reveal many new phenomena behind MS that have not yet been discovered.

show abstract

An Introduction to Information Theory.

Cited by 203 publications

References 0 publications

Holevo bound of entropic uncertainty in Schwarzschild spacetime

Holevo bound of entropic uncertainty in Schwarzschild spacetime

Exploring the Optimization of Autoencoder Design for Imputing Single-Cell RNA Sequencing Data

Assessment of Multivariate Information Transmission in Space-Time-Frequency Domain: A Case Study for EEG Signals

Contact Info

Product

Resources

About