Inferring the directionality of coupling with conditional mutual information

Vejmelka, Martin; Paluš, Milan

doi:10.1103/physreve.77.026214

Cited by 167 publications

(170 citation statements)

References 50 publications

Supporting

Mentioning

169

Contrasting

Unclassified

Order By: Relevance

“…In the latter methods, the probability distribution function estimation depends on the distances between the samples computed using some metric. An example of a metric method is the k-nearest neighbor (kNN [15,25]) algorithm. For more detail on methods of estimation of conditional mutual information and their comparison, see [15].…”

Section: Te Estimationmentioning

confidence: 99%

“…The first is an algorithm based on the discretization of studied variables into Q equiquantal bins (EQQ [24], Q ∈ {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14}) and the second is a k-nearest neighbor algorithm (kNN [15,25], k ∈ {2, 4, 8, 16, 32, 64, 128, 256, 512}).…”

Section: Implementation Detailsmentioning

confidence: 99%

“…TE can be defined in terms of conditional mutual information as shown below, closely following [15]. In particular, we can define that X causes Y if the knowledge of the past of X decreases the uncertainty about Y (above what the knowledge of past of Y and potentially all other relevant confounding variables already informs).…”

Section: Transfer Entropymentioning

confidence: 99%

“…A prominent representative of nonlinear causality assessment is the conditional mutual information [15], especially in the form of transfer entropy [16].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Reliability of Inference of Directed Climate Networks Using Conditional Mutual Information

Hlinka

Hartman

Vejmelka

et al. 2013

Entropy

Self Cite

113

View full text Add to dashboard Cite

Across geosciences, many investigated phenomena relate to specific complex systems consisting of intricately intertwined interacting subsystems. Such dynamical complex systems can be represented by a directed graph, where each link denotes an existence of a causal relation, or information exchange between the nodes. For geophysical systems such as global climate, these relations are commonly not theoretically known but estimated from recorded data using causality analysis methods. These include bivariate nonlinear methods based on information theory and their linear counterpart. The trade-off between the valuable sensitivity of nonlinear methods to more general interactions and the potentially higher numerical reliability of linear methods may affect inference regarding structure and variability of climate networks. We investigate the reliability of directed climate networks detected by selected methods and parameter settings, using a stationarized model of dimensionality-reduced surface air temperature data from reanalysis of 60-year global climate records. Overall, all studied bivariate causality methods provided reproducible estimates of climate causality networks, with the linear approximation showing Entropy 2013, 15 2024 higher reliability than the investigated nonlinear methods. On the example dataset, optimizing the investigated nonlinear methods with respect to reliability increased the similarity of the detected networks to their linear counterparts, supporting the particular hypothesis of the near-linearity of the surface air temperature reanalysis data.

show abstract

Section: Te Estimationmentioning

confidence: 99%

Section: Implementation Detailsmentioning

confidence: 99%

Section: Transfer Entropymentioning

confidence: 99%

“…A prominent representative of nonlinear causality assessment is the conditional mutual information [15], especially in the form of transfer entropy [16].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Reliability of Inference of Directed Climate Networks Using Conditional Mutual Information

Hlinka

Hartman

Vejmelka

et al. 2013

Entropy

Self Cite

113

View full text Add to dashboard Cite

show abstract

“…Transfer entropy has been proposed to distinguish effectively driving and responding elements and to detect asymmetry in the interaction of subsystems. It is it widely applicable because it is model-free and sensitive to nonlinear signal properties [32]. Thus the transfer entropy is able to measure the influences that one measure can exert over another.…”

Section: Primary Contribution Of This Workmentioning

confidence: 99%

Inferring a Drive-Response Network from Time Series of Topological Measures in Complex Networks with Transfer Entropy

2014

Entropy

View full text Add to dashboard Cite

Topological measures are crucial to describe, classify and understand complex networks. Lots of measures are proposed to characterize specific features of specific networks, but the relationships among these measures remain unclear. Taking into account that pulling networks from different domains together for statistical analysis might provide incorrect conclusions, we conduct our investigation with data observed from the same network in the form of simultaneously measured time series. We synthesize a transfer entropy-based framework to quantify the relationships among topological measures, and then to provide a holistic scenario of these measures by inferring a drive-response network. Techniques from Symbolic Transfer Entropy, Effective Transfer Entropy, and Partial Transfer Entropy are synthesized to deal with challenges such as time series being nonstationary, finite sample effects and indirect effects. We resort to kernel density estimation to assess significance of the results based on surrogate data. The framework is applied to study 20 measures across 2779 records in the Technology Exchange Network, and the results are consistent with some existing knowledge. With the drive-response network, we evaluate the influence of each measure by calculating its strength, and cluster them into three classes, i.e., driving measures, responding measures and standalone measures, according to the network communities.

show abstract

Bootstrap rank‐ordered conditional mutual information (broCMI): A nonlinear input variable selection method for water resources modeling

Quilty

Adamowski

Khalil

et al. 2016

Water Resources Research

View full text Add to dashboard Cite

The input variable selection problem has recently garnered much interest in the time series modeling community, especially within water resources applications, demonstrating that information theoretic (nonlinear)‐based input variable selection algorithms such as partial mutual information (PMI) selection (PMIS) provide an improved representation of the modeled process when compared to linear alternatives such as partial correlation input selection (PCIS). PMIS is a popular algorithm for water resources modeling problems considering nonlinear input variable selection; however, this method requires the specification of two nonlinear regression models, each with parametric settings that greatly influence the selected input variables. Other attempts to develop input variable selection methods using conditional mutual information (CMI) (an analog to PMI) have been formulated under different parametric pretenses such as k nearest‐neighbor (KNN) statistics or kernel density estimates (KDE). In this paper, we introduce a new input variable selection method based on CMI that uses a nonparametric multivariate continuous probability estimator based on Edgeworth approximations (EA). We improve the EA method by considering the uncertainty in the input variable selection procedure by introducing a bootstrap resampling procedure that uses rank statistics to order the selected input sets; we name our proposed method bootstrap rank‐ordered CMI (broCMI). We demonstrate the superior performance of broCMI when compared to CMI‐based alternatives (EA, KDE, and KNN), PMIS, and PCIS input variable selection algorithms on a set of seven synthetic test problems and a real‐world urban water demand (UWD) forecasting experiment in Ottawa, Canada.

show abstract

Inferring the directionality of coupling with conditional mutual information

Cited by 167 publications

References 50 publications

Reliability of Inference of Directed Climate Networks Using Conditional Mutual Information

Reliability of Inference of Directed Climate Networks Using Conditional Mutual Information

Inferring a Drive-Response Network from Time Series of Topological Measures in Complex Networks with Transfer Entropy

Bootstrap rank‐ordered conditional mutual information (broCMI): A nonlinear input variable selection method for water resources modeling

Contact Info

Product

Resources

About