From the Entropy to the Statistical Structure of Spike Trains

Gao, Yun; Kontoyiannis, Ioannis; Bienenstock, Elie

doi:10.1109/isit.2006.261864

Cited by 21 publications

(31 citation statements)

References 16 publications

(18 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This work serves, partly, as a more theoretical companion to the experimental work and results presented in [32][33][34]. There, entropy estimators were applied to the spike trains of 28 neurons recorded simultaneously for a one-hour period from the primary motor and dorsal premotor cortices (MI, PMd) of a monkey.…”

Section: Introductionmentioning

confidence: 99%

Estimating the Entropy of Binary Time Series: Methodology, Some Theory and a Simulation Study

Gao¹,

Kontoyiannis

Bienenstock

2008

Entropy

Self Cite

100

104

View full text Add to dashboard Cite

Partly motivated by entropy-estimation problems in neuroscience, we present a detailed and extensive comparison between some of the most popular and effective entropy estimation methods used in practice: The plug-in method, four different estimators based on the Lempel-Ziv (LZ) family of data compression algorithms, an estimator based on the Context-Tree Weighting (CTW) method, and the renewal entropy estimator. METHODOLOGY: Three new entropy estimators are introduced; two new LZ-based estimators, and the "renewal entropy estimator," which is tailored to data generated by a binary renewal process. For two of the four LZ-based estimators, a bootstrap procedure is described for evaluating their standard error, and a practical rule of thumb is heuristically derived for selecting the values of their parameters in practice. THEORY: We prove that, unlike their earlier versions, the two new LZ-based estimators are universally consistent, that is, they converge to the entropy rate for every finite-valued, stationary and ergodic process. An effective method is derived for the accurate approximation of the entropy rate of a finite-state hidden Markov model (HMM) with known distribution. Heuristic calculations are presented and approximate formulas are derived for evaluating the bias and the standard error of each estimator. SIMULATION: All estimators are applied to a wide range of data generated by numerous different processes with varying degrees of dependence and memory. The main conclusions drawn from these experiments include: (i) For all estimators considered, the main source of error is the bias. (ii) The CTW method is repeatedly and consistently seen to provide the most accurate results. (iii) The performance of the LZ-based estimators is often comparable to that of the plug-in method.(iv) The main drawback of the plug-in method is its computational inefficiency; with small word-lengths it fails to detect longer-range structure in the data, and with longer word-lengths the empirical distribution is severely undersampled, leading to large biases.

show abstract

Section: Introductionmentioning

confidence: 99%

Estimating the Entropy of Binary Time Series: Methodology, Some Theory and a Simulation Study

Gao¹,

Kontoyiannis

Bienenstock

2008

Entropy

Self Cite

100

104

View full text Add to dashboard Cite

show abstract

“…Entropy rate can also be viewed as the maximum rate of information creation that can be processed as price changes for studied financial instruments [11] and, as such, as an estimator of entropy production within the studied price formation processes for the purposes of the principle of maximum entropy production. Entropy estimation can be based either on maximum likelihood estimators or estimators based on data compression algorithms [25]. Only the latter can account for long range dependencies in practical applications.…”

Section: Methodsmentioning

confidence: 99%

Multiscale Analysis of the Predictability of Stock Returns

Fiedor¹

2015

Risks

View full text Add to dashboard Cite

Due to the strong complexity of financial markets, economics does not have a unified theory of price formation in financial markets. The most common assumption is the Efficient-Market Hypothesis, which has been attacked by a number of researchers, using different tools. There were varying degrees to which these tools complied with the formal definitions of efficiency and predictability. In our earlier work, we analysed the predictability of stock returns at two time scales using the entropy rate, which can be directly linked to the mathematical definition of predictability. Nonetheless, none of the above-mentioned studies allow any general understanding of how the financial markets work, beyond disproving the Efficient-Market Hypothesis. In our previous study, we proposed the Maximum Entropy Production Principle, which uses the entropy rate to create a general principle underlying the price formation processes. Both of these studies show that the predictability of price changes is higher at the transaction level intraday scale than the scale of daily returns, but ignore all scales in between. In this study we extend these ideas using the multiscale entropy analysis framework to enhance our understanding of the predictability of price formation processes at various time scales.

show abstract

“…We aim to use entropy as an indicator of the degree of predictability associated with a traffic process. The neuroscience community has investigated various estimators for the entropy rate associated with the arrival of neural spikes [Gao et al 2006], that is, the computation of the entropy of a sequence of 1s and 0s. If 1s are associated to a packet arrival, and 0s to no packet arrival for a discrete time interval, a packet flow maps to a spike train.…”

Section: Entropy Estimation Related Workmentioning

confidence: 99%

“…Motivated by the need to study per-application flow traffic, we employ an entropy estimation approach similar to that used in the neuroscience community to study neuron spike trains [Gao et al 2006]. More specifically, we map the packet arrival times of each trace to a binary series and estimate the entropy of this series.…”

Section: Introductionmentioning

confidence: 99%

Characterizing per-application network traffic using entropy

Petkov

Rajagopal

2013

ACM Trans. Model. Comput. Simul.

View full text Add to dashboard Cite

The Internet has been evolving into a more heterogeneous internetwork with diverse new applications imposing more stringent bandwidth and QoS requirements. Already new applications such as YouTube, Hulu, and Netflix are consuming a large fraction of the total bandwidth. We argue that, in order to engineer future internets such that they can adequately cater to their increasingly diverse and complex set of applications while using resources efficiently, it is critical to be able to characterize the load that emerging and future applications place on the underlying network. In this article, we investigate entropy as a metric for characterizing per-flow network traffic complexity. While previous work has analyzed aggregated network traffic, we focus on studying isolated traffic flows. Per-application flow characterization caters to the need of network control functions such as traffic scheduling and admission control at the edges of the network. Such control functions necessitate differentiating network traffic on a per-application basis. The "entropy fingerprints" that we get from our entropy estimator summarize many characteristics of each application's network traffic. Not only can we compare applications on the basis of peak entropy, but we can also categorize them based on a number of other properties of the fingerprints.

show abstract

From the Entropy to the Statistical Structure of Spike Trains

Cited by 21 publications

References 16 publications

Estimating the Entropy of Binary Time Series: Methodology, Some Theory and a Simulation Study

Estimating the Entropy of Binary Time Series: Methodology, Some Theory and a Simulation Study

Multiscale Analysis of the Predictability of Stock Returns

Characterizing per-application network traffic using entropy

Contact Info

Product

Resources

About