This review provides a summary of methods originated in (non-equilibrium) statistical mechanics and information theory, which have recently found successful applications to quantitatively studying complexity in various components of the complex Entropy 2013, 15 4845 system Earth. Specifically, we discuss two classes of methods: (i) entropies of different kinds (e.g., on the one hand classical Shannon and Rényi entropies, as well as non-extensive Tsallis entropy based on symbolic dynamics techniques and, on the other hand, approximate entropy, sample entropy and fuzzy entropy); and (ii) measures of statistical interdependence and causality (e.g., mutual information and generalizations thereof, transfer entropy, momentary information transfer). We review a number of applications and case studies utilizing the above-mentioned methodological approaches for studying contemporary problems in some exemplary fields of the Earth sciences, highlighting the potentials of different techniques.