A real data-driven simulation strategy to select an imputation method for mixed-type trait data

Abstract

Missing observations in trait datasets pose an obstacle for analyses in myriad biological disciplines. Imputation offers an alternative to removing cases with missing values from datasets. Imputation techniques that incorporate phylogenetic information into their estimations have demonstrated improved accuracy over standard techniques. However, previous studies of phylogenetic imputation tools are largely limited to simulations of numerical trait data, with categorical data not evaluated. It also remains to be explored whether the type of genetic data used affects imputation accuracy. We conducted a real data-based simulation study to compare the performance of imputation methods using a mixed-type trait dataset (lizards and amphisbaenians; order: Squamata). Selected methods included mean/mode imputation, k-nearest neighbour, random forests, and multivariate imputation by chained equations (MICE). Known values were removed from a complete-case dataset to simulate different missingness scenarios: missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR). Each method (with and without phylogenetic information derived from mitochondrial and nuclear gene trees) was used to impute the removed values. The performances of the methods were evaluated for each trait and in each missingness scenario. A random forest method supplemented with a nuclear-derived phylogeny performed best overall, and this method was used to impute missing values in the original squamate dataset. Data with imputed values better reflected the characteristics and distributions of the original data compared to the complete-case data. However, phylogeny did not always improve performance for every trait and in every missingness scenario, and caution should be taken when imputing trait data, particularly in cases of extreme bias. Ultimately, these results support the use of a real data-based simulation procedure to select a suitable imputation strategy for a given mixed-type trait dataset. Moreover, they highlight the potential biases that complete-case usage may introduce into analyses.Author summaryThe issue of missing data is problematic in trait datasets as observations for rare or threatened species are often missing disproportionately. When only complete cases are used in an analysis, derived results may be biased. Imputation is an alternative to complete-case analysis and entails filling in the missing values using known observations. It has been demonstrated that including phylogenetic information in the imputation process improves accuracy of predicted values. However, most previous evaluations of imputation methods for trait datasets are limited to numerical, simulated data, with categorical traits not considered. Using a reptile dataset comprised of both numerical and categorical trait data, we employed a real data-based simulation strategy to select an optimal imputation method for the dataset. We evaluated the performance of four different imputation methods across different missingness scenarios (e.g. missing completely at random, values missing disproportionately for smaller species. Results indicate that imputed data better reflected the original dataset characteristics compared to complete-case data; however, the optimal imputation strategy for a given scenario was contingent on missingness scenario and trait type. As imputation performance varies depending on the properties of a given dataset, a real data-based simulation strategy can be used to provide guidance on best imputation practices.

Functional diversity metrics can perform well with highly incomplete data sets

“…Therefore, while our results agree with others that random forest models (as implemented by the missForest R function) are an accurate imputation method for trait data (Johnson et al, 2021), care should be taken to ensure use of imputation is appropriate. Our findings regarding the utility of imputation are only applicable to continuous trait imputation, as the efficacy of categorical traits imputation was not explored (although see May et al, 2023), and to large trait data sets on the scale of hundreds or thousands of species rather than tens. The utility of imputation in tackling missing and biased data has been shown to depend on the correlation between traits, and extent of phylogenetic autocorrelation (Clavel et al, 2015).…”

“…We tested two ways of dealing with the generated incomplete data sets: (1) removal of species with missing data (complete case analysis) and (2) filling data gaps through imputation. We used missForest imputation, implemented through the missForest (Stekhoven & Bühlmann, 2012), due to its demonstrated accuracy (Hong & Lynn, 2020; May et al, 2023; Penone et al, 2014), and fast computation times. Accounting for phylogenetic relatedness between species can improve imputation accuracy (May et al, 2023; Penone et al, 2014).…”

“…We used missForest imputation, implemented through the missForest (Stekhoven & Bühlmann, 2012), due to its demonstrated accuracy (Hong & Lynn, 2020; May et al, 2023; Penone et al, 2014), and fast computation times. Accounting for phylogenetic relatedness between species can improve imputation accuracy (May et al, 2023; Penone et al, 2014). One way of including phylogenetic data is through eigenvectors (Debastiani et al, 2021).…”