BHPMF – a hierarchical Bayesian approach to gap‐filling and trait prediction for macroecology and functional biogeography

Abstract

Aim Functional traits of organisms are key to understanding and predicting biodiversity and ecological change, which motivates continuous collection of traits and their integration into global databases. Such trait matrices are inherently sparse, severely limiting their usefulness for further analyses. On the other hand, traits are characterized by the phylogenetic trait signal, trait-trait correlations and environmental constraints, all of which provide information that could be used to statistically fill gaps. We propose the application of probabilistic models which, for the first time, utilize all three characteristics to fill gaps in trait databases and predict trait values at larger spatial scales.Innovation For this purpose we introduce BHPMF, a hierarchical Bayesian extension of probabilistic matrix factorization (PMF). PMF is a machine learning technique which exploits the correlation structure of sparse matrices to impute missing entries. BHPMF additionally utilizes the taxonomic hierarchy for trait prediction and provides uncertainty estimates for each imputation. In combination with multiple regression against environmental information, BHPMF allows for extrapolation from point measurements to larger spatial scales. We demonstrate the applicability of BHPMF in ecological contexts, using different plant functional trait datasets, also comparing results to taking the species mean and PMF.Main conclusions Sensitivity analyses validate the robustness and accuracy of BHPMF: our method captures the correlation structure of the trait matrix as well as the phylogenetic trait signal -also for extremely sparse trait matrices -and provides a robust measure of confidence in prediction accuracy for each missing entry. The combination of BHPMF with environmental constraints provides a promising concept to extrapolate traits beyond sampled regions, accounting for intraspecific trait variability. We conclude that BHPMF and its derivatives have a high potential to support future trait-based research in macroecology and functional biogeography.

The global spectrum of plant form and function

“…We applied the data selection process as described above (section: Error detection and data quality control). The resulting dataset contained 78% missing entries (gaps), which were filled by Bayesian hierarchical probabilistic matrix factorization (BHPMF) [253][254][255] . The gap-filled dataset was then used to calculate species mean trait values, resulting in a gap-filled dataset for 45,507 species.…”

How to design trait-based analyses of community assembly mechanisms: Insights and guidelines from a literature review

“…Potentially more reliable alternatives to replace missing data would be to use trait values of phylogenetically related species, preferentially after quantification of the degree of phylogenetic signal in each trait (see e.g. Münkemüller et al, 2012;Schrodt et al, 2015). Such imputation could be an alternative to intensive in situ trait measurements representing significant time and financial costs for data collection.…”

Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information

“…Recently, ecologists have made substantial progress in (i) the assessment of the best imputation methods in trait-based applications, (ii) how these methods perform with increasing missingness, (iii) which ecological covariates aid to improve imputations and (iv) how different imputation methods impact the results of trait-based analyses (Pakeman, 2014;Taugourdeau et al, 2014;Penone et al, 2014;Schrodt et al, 2015). Most effort thus far, however, has been directed at imputing species-level trait means and all the abovementioned questions have rarely been assessed on the same dataset.…”

“…The expected declining performance of imputation methods with increasing missingness levels may be trait and dataset dependent (Penone et al, 2014;Taugourdeau et al, 2014). Moreover, the impact of imputations on altering bivariate trait relationships has only been assessed for single relationships (Penone et al, 2014;Schrodt et al, 2015) and not for the multiple relevant relationships within a plant trait dataset. Likewise, there are few studies quantifying how different imputation methods alter the multivariate covariance structure of plant trait datasets .…”