Entropy: from thermodynamics to signal processing

Abstract

Entropy is a concept that remote to the 19th century and it was associated with the work realized by a thermal machine in the context of the Industrial Revolution. The 20th century saw an unprecedented scientific revolution and one of the most important innovations from this time was Information Theory, which also has a concept of Entropy. It can be argued that this is one of the most misused scientific therms and researchers of different areas have been using it wrong. In this paper, a historical background for the evolution of the concept of "entropy" is presented, as well as mathematical proofs and logical arguments for the interconnection of the concept in different areas of science, and how it is related with complexity.