Benchmarking Parametric and Machine Learning Models for Genomic Prediction of Complex Traits

Abstract

The usefulness of genomic prediction in crop and livestock breeding programs has prompted efforts to develop new and improved genomic prediction algorithms, such as artificial neural networks and gradient tree boosting. However, the performance of these algorithms has not been compared in a systematic manner using a wide range of datasets and models. Using data of 18 traits across six plant species with different marker densities and training population sizes, we compared the performance of six linear and six non-linear algorithms. First, we found that hyperparameter selection was necessary for all non-linear algorithms and that feature selection prior to model training was critical for artificial neural networks when the markers greatly outnumbered the number of training lines. Across all species and trait combinations, no one algorithm performed best, however predictions based on a combination of results from multiple algorithms (i.e., ensemble predictions) performed consistently well. While linear and non-linear algorithms performed best for a similar number of traits, the performance of non-linear algorithms vary more between traits. Although artificial neural networks did not perform best for any trait, we identified strategies (i.e., feature selection, seeded starting weights) that boosted their performance to near the level of other algorithms. Our results highlight the importance of algorithm selection for the prediction of trait values.

Potential of Genome-Wide Association Studies and Genomic Selection to Improve Productivity and Quality of Commercial Timber Species in Tropical Rainforest, a Case Study of Shorea platyclados

“…Furthermore, prediction accuracies obtained from all Bayesian and Machine Learning methods were trait-dependent, as found by [60,88]. However, all the Machine Learning methods provided better prediction accuracies for all focal traits of S. platyclados than the Bayesian methods.…”

A Multiple-Trait Bayesian Lasso for Genome-Enabled Analysis and Prediction of Complex Traits

“…It is risky to make sweeping statements arguing in favor of a specific treatment of data, as outcomes are heavily dependent on the biological architecture of the traits considered, and on the data structure as well. The picture emerging from two decades of experience in genome-enabled prediction in the fields of animal and plant breeding is that, in view of the large variability of performance with respect to data structure for any given prediction machine, it is largely futile to categorize methods in terms of expected predictive performance using broad criteria (Morota and Gianola 2014;Gianola and Rosa 2015;Momen et al 2018;Montesinos-López et al 2018a,b, 2019aAzodi et al 2019).…”

A multiple-trait Bayesian Lasso for genome-enabled analysis and prediction of complex traits

“…It is risky to make sweeping statements arguing in favor of a specific treatment of data as outcomes are heavily dependent on the biological architecture of the traits considered, and on the data structure as well. The picture emerging from two decades of experience in genome-enabled prediction in the fields of animal and plant breeding is that is largely futile to categorize methods in terms of expected predictive performance using broad criteria, in view of the large variability of performance with respect to data structure for any given prediction machine (Morota and Gianola 2014; Gianola and Rosa 2015; Momen et al 2018; Montesinos-López et al 2019 a,b,c,d; Azodi et al 2019).…”