Insights into Entropy as a Measure of Multivariate Variability

Information Theory is a branch of mathematics, more specifically probability theory, that studies information quantification. Recently, several researches have been successful with the use of Information Theoretic Learning (ITL) as a new technique of unsupervised learning. In these works, information measures are used as criterion of optimality in learning. In this article, we will analyze a still unexplored aspect of these information measures, their dynamic behavior. Autoregressive models (linear and non-linear) will be used to represent the dynamics in information measures. As a source of dynamic information, videos with different characteristics like fading, monotonous sequences, etc., will be used.

Section: Information Theoretic Learning (Itl)mentioning

confidence: 99%

An Analysis of Information Dynamic Behavior Using Autoregressive Models

Oliveira

Neto

Martins

2017

“…Entropy can be used to quantify uncertainty, complexity, randomness, and regularity [14]. In recent decades, entropy has come to be applied very broadly [15].…”

Section: Trajectory Entropymentioning

confidence: 99%

Sparse Trajectory Prediction Based on Multiple Entropy Measures

Zhang

Liu

Xia

et al. 2016

Trajectory prediction is an important problem that has a large number of applications. A common approach to trajectory prediction is based on historical trajectories. However, existing techniques suffer from the "data sparsity problem". The available historical trajectories are far from enough to cover all possible query trajectories. We propose the sparsity trajectory prediction algorithm based on multiple entropy measures (STP-ME) to address the data sparsity problem. Firstly, the moving region is iteratively divided into a two-dimensional plane grid graph, and each trajectory is represented as a grid sequence with temporal information. Secondly, trajectory entropy is used to evaluate trajectory's regularity, the L-Z entropy estimator is implemented to calculate trajectory entropy, and a new trajectory space is generated through trajectory synthesis. We define location entropy and time entropy to measure the popularity of locations and timeslots respectively. Finally, a second-order Markov model that contains a temporal dimension is adopted to perform sparse trajectory prediction. The experiments show that when trip completed percentage increases towards 90%, the coverage of the baseline algorithm decreases to almost 25%, while the STP-ME algorithm successfully copes with it as expected with only an unnoticeable drop in coverage, and can constantly answer almost 100% of query trajectories. It is found that the STP-ME algorithm improves the prediction accuracy generally by as much as 8%, 3%, and 4%, compared to the baseline algorithm, the second-order Markov model (2-MM), and sub-trajectory synthesis (SubSyn) algorithm, respectively. At the same time, the prediction time of STP-ME algorithm is negligible (10 µs), greatly outperforming the baseline algorithm (100 ms).

“…Entropy is a popular measure of uncertainty and disorder and has been successfully applied in adaptive signal processing [24][25][26]. It also has been applied in power systems for quite a long time [27][28][29].…”

Section: Definition Of Wavelet Entropymentioning

confidence: 99%

Recognition of Traveling Surges in HVDC with Wavelet Entropy

Luo

Lin

Zhou

et al. 2017

Abstract:Traveling surges are commonly adopted in protection devices of high-voltage direct current (HVDC) transmission systems. Lightning strikes also can produce large-amplitude traveling surges which lead to the malfunction of relays. To ensure the reliable operation of protection devices, recognition of traveling surges must be considered. Wavelet entropy, which can reveal time-frequency distribution features, is a potential tool for traveling surge recognition. In this paper, the effectiveness of wavelet entropy in characterizing traveling surges is demonstrated by comparing its representations of different kinds of surges and discussing its stability with the effects of propagation distance and fault resistance. A wavelet entropy-based recognition method is proposed and tested by simulated traveling surges. The results show wavelet entropy can discriminate fault traveling surges with a good recognition rate.