The Measurement of Information Transmitted by a Neural Population: Promises and Challenges

Crumiller, Marshall; Knight, Bruce W.; Kaplan, Ehud

doi:10.3390/e15093507

Cited by 10 publications

(10 citation statements)

References 43 publications

(50 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…I also studied the interesting case when instead of H , I used a well known unimodal map

[ 45 ] which is, in fact, close ( Figure 1 ) to H in the supremum norm [ 46 ]. This idea is along the research direction related to generalized concepts of entropy developed, starting from Renyai [ 47 ], by many authors [ 48 , 49 , 50 , 51 , 52 ]. Figure 1 shows the approximation of entropy ( 11 ) by polynomials: unimodal map (black dash line) and 10 first terms in the Taylor series of H (gray dash-dot line).…”

Section: Theoretical Background and Methodsmentioning

confidence: 99%

Signal Fluctuations and the Information Transmission Rates in Binary Communication Channels

Pręgowska

2021

Entropy

View full text Add to dashboard Cite

In the nervous system, information is conveyed by sequence of action potentials, called spikes-trains. As MacKay and McCulloch suggested, spike-trains can be represented as bits sequences coming from Information Sources (IS). Previously, we studied relations between spikes’ Information Transmission Rates (ITR) and their correlations, and frequencies. Now, I concentrate on the problem of how spikes fluctuations affect ITR. The IS are typically modeled as stationary stochastic processes, which I consider here as two-state Markov processes. As a spike-trains’ fluctuation measure, I assume the standard deviation σ, which measures the average fluctuation of spikes around the average spike frequency. I found that the character of ITR and signal fluctuations relation strongly depends on the parameter s being a sum of transitions probabilities from a no spike state to spike state. The estimate of the Information Transmission Rate was found by expressions depending on the values of signal fluctuations and parameter s. It turned out that for smaller s<1, the quotient ITRσ has a maximum and can tend to zero depending on transition probabilities, while for 1<s, the ITRσ is separated from 0. Additionally, it was also shown that ITR quotient by variance behaves in a completely different way. Similar behavior was observed when classical Shannon entropy terms in the Markov entropy formula are replaced by their approximation with polynomials. My results suggest that in a noisier environment (1<s), to get appropriate reliability and efficiency of transmission, IS with higher tendency of transition from the no spike to spike state should be applied. Such selection of appropriate parameters plays an important role in designing learning mechanisms to obtain networks with higher performance.

show abstract

“…I also studied the interesting case when instead of H , I used a well known unimodal map

Section: Theoretical Background and Methodsmentioning

confidence: 99%

Signal Fluctuations and the Information Transmission Rates in Binary Communication Channels

Pręgowska

2021

Entropy

View full text Add to dashboard Cite

show abstract

“…We also studied the interesting case when instead of H we used famous unimodal map U (p) = 4p(1 − p) [35] which is, in fact, close ( Figure 1) to H in the supremum norm [36]. This idea is along the research direction related to generalized concepts of entropy developed, starting from Renyai [37], by many authors [38,39,40,41,42]. Figure 1 shows the approximation of…”

Section: Information Sources Bernoulli Process Casementioning

confidence: 99%

Signal Fluctuations and the Information Transmission Rates in Binary Communication Channels

Pręgowska¹

2020

Preprint

View full text Add to dashboard Cite

(1) Background: In nervous system information is conveyed by a sequence of action potentials, called spikes-trains. As MacKay and McCulloch proposed, spike-trains can be represented as bits sequences coming from Information Sources. Previously, we studied relations between Information Transmission Rates (ITR) carried out by the spikes, their correlations, and frequencies. Here, we concentrate on the problem of how spikes fluctuations affect ITR. (2) Methods: The Information Theory Method developed by Shannon is applied. Information Sources are modeled as stationary stochastic processes. We assume such sources as two states Markov processes. As a spike-trains' fluctuation measure, we consider the Standard Deviation sigma, which, in fact, measures average fluctuation of spikes around the average spike frequency. (3) Results: We found that character of ITR and signal fluctuations relation strongly depends on the parameter s which is a sum of transitions probabilities from no spike state to spike state and vice versa. It turned out that for smaller s (s<1) the quotient ITR\sigma has a maximum and can tend to zero depending on transition probabilities. While for s large enough (1<s) the ITR\sigma is separated from 0 for each s. Similar behavior was observed also when we replaced Shannon entropy terms in Markov entropy formula by their approximation with polynomials. We also show that the ITR quotient by Variance behaves in a completely different way. (4) Conclusions: Our results show that for large transition parameter s the Information Transmission Rate by sigma will never decrease to zero. Specifically, for 1<s<1.7 the ITR will be always, independently on transition probabilities which form this s, above the level of fluctuations, i.e. in this case we have sigma<ITR. Thus, we conclude that in a more noisy environment, to get appropriate reliability and efficiency of transmission, Information Sources with higher tendency of transition from the state no spike to spike state and vice versa should be applied.

show abstract

“…This limit exists if and only if the stochastic process Z is stationary. 17 The most commonly used definition of Firing Rate refers to the temporal average [19][20][21] and is defined as…”

Section: Information Transmission and Firing Ratesmentioning

confidence: 99%

How Far can Neural Correlations Reduce Uncertainty? Comparison of Information Transmission Rates for Markov and Bernoulli Processes

Pręgowska

Kaplan

Szczepański

2019

Int. J. Neur. Syst.

Self Cite

View full text Add to dashboard Cite

The nature of neural codes is central to neuroscience. Do neurons encode information through relatively slow changes in the emission rates of individual spikes (rate code), or by the precise timing of every spike (temporal codes)? Here we compare the loss of information due to correlations for these two possible neural codes. The essence of Shannon's definition of information is to combine information with uncertainty: the higher the uncertainty of a given event, the more information is conveyed by that event. Correlations can reduce uncertainty or the amount of information, but by how much? In this paper we address this question by a direct comparison of the information per symbol conveyed by the words coming from a binary Markov source (temporal codes) with the information per symbol coming from the corresponding Bernoulli source (uncorrelated, rate code source). In a previous paper we found that a crucial role in the relation between Information Transmission Rates (IT R) and Firing Rates is played by a parameter s, which is the sum of transitions probabilities from the no-spike-state to the spike-state and vice versa. It turned out that also in this case a crucial role is played by the same parameter s. We found bounds of the quotient of IT Rs for these sources, i.e. this quotient's minimal and maximal values. Next, making use of the entropy grouping axiom, we determined the loss of information in a Markov source in relation to its corresponding Bernoulli source for a given length of word. Our results show that in practical situations in the case of correlated signals the loss of information is relatively small, thus temporal codes, which are more energetically efficient, can replace the rate code effectively. These phenomena were confirmed by experiments.

show abstract

The Measurement of Information Transmitted by a Neural Population: Promises and Challenges

Cited by 10 publications

References 43 publications

Signal Fluctuations and the Information Transmission Rates in Binary Communication Channels

Signal Fluctuations and the Information Transmission Rates in Binary Communication Channels

Signal Fluctuations and the Information Transmission Rates in Binary Communication Channels

How Far can Neural Correlations Reduce Uncertainty? Comparison of Information Transmission Rates for Markov and Bernoulli Processes

Contact Info

Product

Resources

About