Handling missing values in trait data

Abstract

Aim Trait data are widely used in ecological and evolutionary phylogenetic comparative studies, but often values are not available for all species of interest. Traditionally, researchers have excluded species without data from analyses, but estimation of missing values using imputation has been proposed as a better approach. However, imputation methods have largely been designed for randomly missing data, whereas trait data are often not missing at random (e.g., more data for bigger species). Here, we evaluate the performance of approaches for handling missing values when considering biased datasets. Location Any. Time period Any. Major taxa studied Any. Methods We simulated continuous traits and separate response variables to test the performance of nine imputation methods and complete‐case analysis (excluding missing values from the dataset) under biased missing data scenarios. We characterized performance by estimating the error in imputed trait values (deviation from the true value) and inferred trait–response relationships (deviation from the true relationship between a trait and response). Results Generally, Rphylopars imputation produced the most accurate estimate of missing values and best preserved the response–trait slope. However, estimates of missing data were still inaccurate, even with only 5% of values missing. Under severe biases, errors were high with every approach. Imputation was not always the best option, with complete‐case analysis frequently outperforming Mice imputation and, to a lesser degree, BHPMF imputation. Mice, a popular approach, performed poorly when the response variable was excluded from the imputation model. Main conclusions Imputation can handle missing data effectively in some conditions but is not always the best solution. None of the methods we tested could deal effectively with severe biases, which can be common in trait datasets. We recommend rigorous data checking for biases before and after imputation and propose variables that can assist researchers working with incomplete datasets to detect data biases and minimize errors.

Intraspecific trait variation and species functional turnover in successional tropical forests: assessing gap-filling for community weighted means

“…This likely re ects both functional trait variation maximized by the sampling design because of forest type and actual ITV among tropical trees. Concerning our second research question, on the sensitivity of patterns of CWMs to gap-lling via phylogenetic trait imputation, we nd support for the PhyloPars method, which is in congruence with some recent work (Johnson et al 2021). Inference was not unchanged quantitively by trait imputation (i.e., accepting the null hypothesis as outlined in the introduction), however patters were strikingly similar (i.e., changed very little qualitatively).…”

“…To validate the results regarding intraspeci c variability effects using AWMs, we employed an approach to gap-ll trait data for unsampled taxa. As we have explained, PhyloPars (Bruggeman et al 2009) appears to be one of the best tools available for gap lling functional traits (Penone et al 2014, Johnson et al 2021. Additionally, due to the high degree of phylogenetic-conservatism in certain root traits (e.g., root diameter), PhyloPars is particularly appropriate.…”

“…Two of the next most-desirable methods include Bayesian hierarchal probabilistic matrix factorization (BHPMF, Schrodt et al 2015), which can use a hierarchical taxonomic structure (e.g., genera within families within orders) to infer phylogenetic structure in the imputation of missing trait values, and PhyloPars (Bruggeman et al 2009, Goolsby et al 2017, which uses phylogenetic generalized linear models (pGLM) to impute missing trait values, thereby directly accounting for species relatedness via branch lengths. A recent study compared the three methods, concluding that PhyloPars outperforms MICE and BHPMF for trait imputation (Johnson et al 2021).…”

Erosion of global functional diversity across the tree of life

“…The accuracy of trait imputation procedures depends on factors such as the proportion of missing observations and the potential biases in trait measurements. Although some studies have analyzed the performance of imputation methods in the context of retrieving individual trait values (25,26), the goal of our imputations was rather to characterize the position of species in the corresponding trait space established using PCA based on the individual traits. The facts that (i) many traits are strongly correlated and (ii) evolutionarily closely related species tend to be close in the functional space effectively mean that the position of species in the PCA should be easier to estimate than the individual trait values.…”

Pitfalls of ignoring trait resolution when drawing conclusions about ecological processes

“…unequal distribution of missing data across species within a taxonomic group, trait values, etc. ; Johnson et al., 2020). Thus, in cases where the species coverage of a continuous trait of interest is still prohibitively low for typical imputation, methods that take advantage of the structure of the data via phylogeny, spatial trait variation, or correlations with other, well‐sampled traits (Kim et al., 2018; Schrodt et al., 2015) could be adapted to probabilistically refine categorical trait assignments of species.…”