Permutation Entropy: Enhancing Discriminating Power by Using Relative Frequencies Vector of Ordinal Patterns Instead of Their Shannon Entropy

Abstract

Many measures to quantify the nonlinear dynamics of a time series are based on estimating the probability of certain features from their relative frequencies. Once a normalised histogram of events is computed, a single result is usually derived. This process can be broadly viewed as a nonlinear I R n mapping into I R , where n is the number of bins in the histogram. However, this mapping might entail a loss of information that could be critical for time series classification purposes. In this respect, the present study assessed such impact using permutation entropy (PE) and a diverse set of time series. We first devised a method of generating synthetic sequences of ordinal patterns using hidden Markov models. This way, it was possible to control the histogram distribution and quantify its influence on classification results. Next, real body temperature records are also used to illustrate the same phenomenon. The experiments results confirmed the improved classification accuracy achieved using raw histogram data instead of the PE final values. Thus, this study can provide a very valuable guidance for the improvement of the discriminating capability not only of PE, but of many similar histogram-based measures.

Slope Entropy: A New Time Series Complexity Estimator Based on Both Symbolic Patterns and Amplitude Information

“…The classification accuracy achieved with SlopEn was the highest in all cases tested ( Table 1 ). Since the datasets were chosen from previous works where PE exhibited some limitations due to its inability to include amplitude information [ 33 , 42 ], its results were the worst of the three metrics, as expected. PE only found significant differences for EEG and EMG records.…”

“…Two of the most used entropy quantification methods were included in the experiments for comparative purposes: PE, as a good representative of ordinal–based approaches, and SampEn, based on amplitude differences. The experimental dataset included usual biomedical records in classification studies: EEG and RR records, records where PE achieved very good classification accuracy, EMG records, and synthetic and real records where PE has failed because amplitude information was a key distinguishing feature: Gaussian and uniform random noise [ 42 ], and energy consumption records [ 33 ]. This way, the study was also not constrained to just biomedical records.…”

“…There is a clear synthetic case where PE failed to find differences between two classes: random time series with Gaussian or uniform amplitude distributions. This is a representative example of what happens when classes under analysis have the same temporal correlations but differ in amplitude: PE discriminating power gets lost [ 42 ]. A dataset of this case was included in the experiments in order to find out if SlopEn was capable of overcoming this known weakness of PE.…”

Using the Information Provided by Forbidden Ordinal Patterns in Permutation Entropy to Reinforce Time Series Discrimination Capabilities

“…First, the addition of synthetic databases was considered, since this kind of records is also very useful for characterising the performances of methods under more controlled conditions. In a very recent paper [ 32 ], we proposed to use a hidden Markov model to create synthetic records based on transition probabilities of their ordinal patterns of length . This is a very suitable tool to create a synthetic dataset for the present study, since the main difference between PE1 and PE2 is the use of the number of actual ordinal patterns found.…”

“… being the transition probability between consecutive states and at time t , , and the following correspondence between model states and ordinal patterns: , , , , , and , 100 records of two synthetic classes were generated using this model. For one class, the transition probabilities were , and for the second class , probabilities defined as in [ 32 ]. Therefore, each class penalised a different transition, and that would impact the number of patterns found at and beyond in a different way for each class, since the model is not symmetric (see details in [ 32 ]).…”