Benchmarking algorithms for genomic prediction of complex traits

Abstract

The usefulness of Genomic Prediction (GP) in crop and livestock breeding programs has led to efforts to develop new and improved GP approaches including non-linear algorithm, such as artificial neural networks (ANN) (i.e. deep learning) and gradient tree boosting. However, the performance of these algorithms has not been compared in a systematic manner using a wide range of GP datasets and models. Using data of 18 traits across six plant species with different marker densities and training population sizes, we compared the performance of six linear and five non-linear algorithms, including ANNs. First, we found that hyperparameter selection was critical for all non-linear algorithms and that feature selection prior to model training was necessary for ANNs when the markers greatly outnumbered the number of training lines. Across all species and trait combinations, no one algorithm performed best, however predictions based on a combination of results from multiple GP algorithms (i.e. ensemble predictions) performed consistently well. While linear and non-linear algorithms performed best for a similar number of traits, the performance of non-linear algorithms vary more between traits than that of linear algorithms. Although ANNs did not perform best for any trait, we identified strategies (i.e. feature selection, seeded starting weights) that boosted their performance near the level of other algorithms. These results, together with the fact that even small improvements in GP performance could accumulate into large genetic gains over the course of a breeding program, highlights the importance of algorithm selection for the prediction of trait values.

<strong>Factors Influencing Genomic Prediction Accuracies of Tropical Maize Resistance to Fall Armyworm and Weevils</strong>

“…In this study, several GP models that included statistical and Machine Learning algorithms from parametric, semi-parametric, and nonparametric approaches were used to predict FAW and MW resistance traits. These GP algorithms, as expected, performed differently on the different traits although the predictive variations were generally minimal, especially when large TS were involved, similar to earlier model benchmarking reports (101,102). Bayesian models (parametric: BLR and BRR, and semiparametric: RKHS) performed better on MW traits, GWL, AP, and AK, while nonparametric Machine Learning algorithms (missForest, here) and, to a lesser extent, linear mixed model (especially in the PBTS approach), achieved the highest PAs on FAW datasets.…”

Genomic Prediction of Tropical Maize Resistance to Fall Armyworm and Weevils: Genomic Selection Should Focus on Effective Training Set Determination

“…In this study, several GP models including statistical and Machine Learning algorithms, parametric, semi-parametric, and nonparametric approaches were used to predict FAW and MW resistance traits. These GP algorithms, as expected, performed differently on the different traits although the predictive variations were generally minimal, especially when large TS were involved, similar to earlier model benchmarking reports (112,113). Bayesian models (parametric: BLR and BRR, and semi-parametric: RKHS) performed better on MW traits, GWL, AP, and AK, while nonparametric Machine Learning algorithms (missForest) and, to a lesser extent, linear mixed model (especially in the PBTS approach), achieved the highest PAs on FAW datasets.…”

Exploring Deep Learning for Complex Trait Genomic Prediction in Polyploid Outcrossing Species

“…pattern recognition (Drayer and Brox, 2014;Liang and Hu, 2015;Işin et al, 2016;Badrinarayanan et al, 2017) and natural language processing (NLP) (Deng and Liu, 2018). The DL implementation in regression tasks is less abundant and the benefit of using these methods remains uncertain (Bellot et al, 2018;Montesinos-López et al, 2018a;Azodi et al, 2019). Most GP problems fall into the regression task due to the complex nature of quantitative traits (MacKay, 2009).…”