Entropy Measures vs. Kolmogorov Complexity

Teixeira, Andréia; Matos, Armando B.; Souto, André; Antunes, Luís

doi:10.3390/e13030595

Cited by 40 publications

(22 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By using unsupervised learning, the authors showed that anomalies can be successfully clustered. Wagner and Plattner [7] made use of the Kolmogorov Complexity (related to Shannon entropy) [68,69] in order to detect worms in network traffic. Their work mostly focuses on implementation aspects and scalability and does not propose any specific analysis techniques.…”

Section: Detection Via Feature Distributionsmentioning

confidence: 99%

An Entropy-Based Network Anomaly Detection Method

Bereziński

Jasiul

Szpyrka

2015

Entropy

157

View full text Add to dashboard Cite

Data mining is an interdisciplinary subfield of computer science involving methods at the intersection of artificial intelligence, machine learning and statistics. One of the data mining tasks is anomaly detection which is the analysis of large quantities of data to identify items, events or observations which do not conform to an expected pattern. Anomaly detection is applicable in a variety of domains, e.g., fraud detection, fault detection, system health monitoring but this article focuses on application of anomaly detection in the field of network intrusion detection.The main goal of the article is to prove that an entropy-based approach is suitable to detect modern botnet-like malware based on anomalous patterns in network. This aim is achieved by realization of the following points: (i) preparation of a concept of original entropy-based network anomaly detection method, (ii) implementation of the method, (iii) preparation of original dataset, (iv) evaluation of the method.

show abstract

Section: Detection Via Feature Distributionsmentioning

confidence: 99%

An Entropy-Based Network Anomaly Detection Method

Bereziński

Jasiul

Szpyrka

2015

Entropy

157

View full text Add to dashboard Cite

show abstract

“…Adonai Sant'Anna [23] directly determines the algorithmic complexity of a thermal system as its entropy. Andreia Teixeira et al [24] considers Kolmogorov complexity and Shannon entropy as conceptually different measures, but convincingly proves that for any recursive probability distribution, the expected value of complexity is equal to the entropy up to a constant. Peter Grunwald and Paul Vitanyi [20] point out that K(x) is not a recursive function, and Kolmogorov complexity in general is not calculable in the sense that there is no a computer program that could calculate K in the Equation (3) for any description x, although in simpler and special cases, it may be calculated, while for more complex cases, it could be semi-calculated (i.e., approximately estimated).…”

Section: Algorithmic Complexity and Its Similarity To Entropymentioning

confidence: 99%

Entropy, Information and Complexity or Which Aims the Arrow of Time?

Mikhailovsky

Levich

2015

Entropy

View full text Add to dashboard Cite

Abstract:In this article, we analyze the interrelationships among such notions as entropy, information, complexity, order and chaos and show using the theory of categories how to generalize the second law of thermodynamics as a law of increasing generalized entropy or a general law of complification. This law could be applied to any system with morphisms, including all of our universe and its subsystems. We discuss how such a general law and other laws of nature drive the evolution of the universe, including physicochemical and biological evolutions. In addition, we determine eliminating selection in physicochemical evolution as an extremely simplified prototype of natural selection. Laws of nature do not allow complexity and entropy to reach maximal values by generating structures. One could consider them as a kind of "breeder" of such selection.

show abstract

“…Despite this theoretical nice property, the practical use of this measure is limited not only by its incomputability, but also by the imprecision of the compressor, when used in its more feasible form. On the contrary, its Shannon entropy version [50], [52], [61] gives better results. The latter, that deals with PDFs instead of strings of symbols, is strictly related to the concept of MI.…”

Section: Perception-based Similarity Metricmentioning

confidence: 97%

An Improvement of Kernel-Based Object Tracking Based on Human Perception

Bruni

Vitulano

2014

IEEE Trans. Syst. Man Cybern, Syst.

View full text Add to dashboard Cite

The objective of the paper is to embed perception rules into the kernel-based target tracking algorithm and to evaluate to what extent these rules are able to improve the original tracking algorithm, without any additional computational cost. To this aim, the target is represented through features that are related to its visual appearance; then, it is tracked in subsequent frames using a metric that, again, correlates well with the human visual perception (HVP). The use of HVP rules are twofold advantageous: it allows us to both increase tracking efficacy and considerably reduce the computational cost of the tracking process-thanks to the reduced size of the perceptual feature space. Various tests on video sequences have shown the stability and the robustness of the proposed framework, also in the presence of both other moving objects and partial or complete target occlusion in a limited number of subsequent frames.

show abstract

Entropy Measures vs. Kolmogorov Complexity

Cited by 40 publications

References 12 publications

An Entropy-Based Network Anomaly Detection Method

An Entropy-Based Network Anomaly Detection Method

Entropy, Information and Complexity or Which Aims the Arrow of Time?

An Improvement of Kernel-Based Object Tracking Based on Human Perception

Contact Info

Product

Resources

About