Revisiting the concept of a symmetric index of agreement for continuous datasets

Abstract

Quantifying how close two datasets are to each other is a common and necessary undertaking in scientific research. The Pearson product-moment correlation coefficient r is a widely used measure of the degree of linear dependence between two data series, but it gives no indication of how similar the values of these series are in magnitude. Although a number of indexes have been proposed to compare a dataset with a reference, only few are available to compare two datasets of equivalent (or unknown) reliability. After a brief review and numerical tests of the metrics designed to accomplish this task, this paper shows how an index proposed by Mielke can, with a minor modification, satisfy a series of desired properties, namely to be adimensional, bounded, symmetric, easy to compute and directly interpretable with respect to r. We thus show that this index can be considered as a natural extension to r that downregulates the value of r according to the bias between analysed datasets. The paper also proposes an effective way to disentangle the systematic and the unsystematic contribution to this agreement based on eigen decompositions. The use and value of the index is also illustrated on synthetic and real datasets.

Biophysics and vegetation cover change: a process-based evaluation framework for confronting land surface models with satellite observations

“…15 This is not the case when using a coefficient of determination R 2 from a standard regression of Y on X, which differs from that of a regression of X on Y. Beyond being symmetric, the index of agreement λ that we use is also dimensionless, bounded (between 0 for no agreement to 1 for perfect agreement) and easy to compute (Duveiller et al, 2016). Furthermore, its interpretation is relatively intuitive for practitioners since there are no additive or multiplicative bias contributing to the disagreement between X and Y, its value becomes that of the correlation coefficient r. For two sets, X and Y, each containing 20 n values, the index is defined as:…”

Local‐regional similarity in drylands increases during multiyear wet and dry periods and in response to extreme events

“…To compare JRN PPT to regional values, we correlated the 5th to 95th percentiles of JRN precipitation anomaly (Pa) to the same percentiles across distances from JRN (from 0 to 50 km to 0 to 500 km). Reported r values for each percentile of Pa were calculated using a modified version of Mielke's r from Duveiller et al (2015), which reduces the r correlation coefficient in response to additive and multiplicative biases, instead of major-axis regressions or slope-intercept techniques, which are less-adept at accounting for these biases (Mesple et al 1996, Pineiro et al 2008. As a result, our reported r values are lower than more typically used r values, but provide a more complete correlation assessment with fewer biases.…”

“…Reported r values for each percentile of Pa were calculated using a modified version of Mielke's r from Duveiller et al. (), which reduces the r correlation coefficient in response to additive and multiplicative biases, instead of major‐axis regressions or slope–intercept techniques, which are less‐adept at accounting for these biases (Mesple et al. , Pineiro et al.…”

The Implication of Different Sets of Climate Variables on Regional Maize Yield Simulations

“…The yield from all the simulation units ( Figure 3) in each country was averaged to obtain a representative value for a specific year. In this study, we used 'Modified Mielke approach' (Equation (4)), a permutation-based index, to evaluate the agreement between the two continuous datasets (in our case, namely Approach 1 and 2) proposed by [43]. We used this approach to quantify how close the datasets in Approach 1 and 2 are to each other as this not only measures the degree of dependence between the two data series but also estimates how similar in magnitude these series' values are.…”