Identifying the Coupling Structure in Complex Systems through the Optimal Causation Entropy Principle

Abstract

Inferring the coupling structure of complex systems from time series data in general by means of statistical and information-theoretic techniques is a challenging problem in applied science. The reliability of statistical inferences requires the construction of suitable information-theoretic measures that take into account both direct and indirect influences, manifest in the form of information flows, between the components within the system. In this work, we present an application of the optimal causation entropy (oCSE) principle to identify the coupling structure of a synthetic biological system, the repressilator. Specifically, when the system reaches an equilibrium state, we use a stochastic perturbation approach to extract time series data that approximate a linear stochastic process. Then, we present and jointly apply the aggregative discovery and progressive removal algorithms based on the oCSE principle to infer the coupling structure of the system from the measured data. Finally, we show that the success rate of our coupling inferences not only improves with the amount of available data, but it also increases with a higher frequency of sampling and is especially immune to false positives.

Thermodynamic aspects of information transfer in complex dynamical systems

“…(3), (9), and (25) within the Liang-Kleeman approach correspond to Eqs. (13), (21), and (19) within the HorowitzEsposito approach, respectively.…”

“…systems is justified by the important role that information transfer analysis has in detecting asymmetry in the interaction of subsystems [1], in predicting the weather [6], in controlling a system [7,8], in inferring causal structures [9,10], and, from a more conceptual standpoint, in investigating the thermodynamics of Maxwell's demon [11][12][13].…”

Minimum Sample Size for Reliable Causal Inference Using Transfer Entropy

“…Recently, sophisticated algorithms using transfer entropy have improved causal detection [8,9]. However, even these advanced methods require sufficient sample points to avoid false positives due to data-shortage bias.…”

Editorial Comment on the Special Issue of “Information in Dynamical Systems and Complex Systems”

“…In the paper "Identifying Coupling Structure in Complex Systems through the Optimal Causation Entropy Principle" by Sun, Cafaro, and Bollt herein [24], the authors discussed the general problem of causality inference in complex systems from time series data based on information-theoretic tools. In particular, the authors reviewed a new theoretical quantity called Causation Entropy introduced in [25] and subsequent developments in [26].…”