Deep Phenotyping: Deep Learning for Temporal Phenotype/Genotype Classification

Abstract

High resolution and high throughput, genotype to phenotype studies in plants are underway to accelerate breeding of climate ready crops. Complex developmental phenotypes are observed by imaging a variety of accessions in different environment conditions, however extracting the genetically heritable traits is challenging. In the recent years, deep learning techniques and in particular Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs) and Long-Short Term Memories (LSTMs), have shown great success in visual data recognition, classification, and sequence learning tasks. In this paper, we proposed a CNN-LSTM framework for plant classification of various genotypes. Here, we exploit the power of deep CNNs for joint feature and classifier learning, within an automatic phenotyping scheme for genotype classification. Further, plant growth variation over time is also important in phenotyping their dynamic behavior. This was fed into the deep learning framework using LSTMs to model these temporal cues for different plant accessions. We generated a replicated dataset of four accessions of Arabidopsis and carried out automated phenotyping experiments. The results provide evidence of the benefits of our approach over using traditional hand-crafted image analysis features and other genotype classification frameworks. We also demonstrate that temporal information further improves the performance of the phenotype classification system.

Applications for deep learning in ecology

“…papers, 46 implemented at least one deep learning model, with one implementing two -a CNN and a RNN 18 . The remaining papers only mentioned or discussed the use of deep learning for ecological studies.…”

A deep learning-based approach for high-throughput hypocotyl phenotyping

“…In recent years, the introduction of deep learning and convolutional neural networks revolutionized computer vision-based research, making the automation of various tasks and precise high-throughput phenotyping available for many disciplines. In plant biology, several advances have been made with these methods regarding qualitative phenotyping (Pound et al, 2017;Namin et al, 2018;Pineda et al, 2018;Singh et al, 2018;Ramcharan et al, 2019). With these tools however, quantitative phenotypic traits can also be assessed as we demonstrated in this work.…”

“…This makes its performance exceptionally good, and given enough data, a well-trained neural network can generalize for a wide range of datasets. For plant phenotyping, these developments yielded advances for instance in trait identification and genotype/phenotype classification (Pound et al, 2017;Namin et al, 2018).…”

Deep Learning for Multi-task Plant Phenotyping

“…Consequently this approach is relatively slow, and lacks context due to the small cropped window sizes. Another approach has used a CNN-LSTM framework to classify plants into genotype [23]. The use of the LSTM is interesting, as it is used to improve classification over time; the author's hypothesise is that growth rate is an important factor in determining genotype.…”

“…Most recently, deep learning promises a step-change in the performance of many imagebased systems (e.g. [18], [23]), but adoption by the plant phenotyping community is still in its infancy.…”