Deep phenotyping: deep learning for temporal phenotype/genotype classification

Abstract

BackgroundHigh resolution and high throughput genotype to phenotype studies in plants are underway to accelerate breeding of climate ready crops. In the recent years, deep learning techniques and in particular Convolutional Neural Networks (CNNs), Recurrent Neural Networks and Long-Short Term Memories (LSTMs), have shown great success in visual data recognition, classification, and sequence learning tasks. More recently, CNNs have been used for plant classification and phenotyping, using individual static images of the plants. On the other hand, dynamic behavior of the plants as well as their growth has been an important phenotype for plant biologists, and this motivated us to study the potential of LSTMs in encoding these temporal information for the accession classification task, which is useful in automation of plant production and care.MethodsIn this paper, we propose a CNN-LSTM framework for plant classification of various genotypes. Here, we exploit the power of deep CNNs for automatic joint feature and classifier learning, compared to using hand-crafted features. In addition, we leverage the potential of LSTMs to study the growth of the plants and their dynamic behaviors as important discriminative phenotypes for accession classification. Moreover, we collected a dataset of time-series image sequences of four accessions of Arabidopsis, captured in similar imaging conditions, which could be used as a standard benchmark by researchers in the field. We made this dataset publicly available.ConclusionThe results provide evidence of the benefits of our accession classification approach over using traditional hand-crafted image analysis features and other accession classification frameworks. We also demonstrate that utilizing temporal information using LSTMs can further improve the performance of the system. The proposed framework can be used in other applications such as in plant classification given the environment conditions or in distinguishing diseased plants from healthy ones.

Applications for deep learning in ecology

“…N/A includes studies that deal with but do not implement deep learning methods. Note that one study (Taghavi Namin, Esmaeilzadeh, Najafi, Brown, & Borevitz, ) was counted twice as it implemented a combination of CNN and RNN. The list of selected papers used ( n = 87) to reproduce the figure can be found in supporting information 1…”

Exploring Deep Learning for Complex Trait Genomic Prediction in Polyploid Outcrossing Species

“…Deep learning (DL) techniques comprise a heterogeneous collection of machine learning algorithms which have excelled at many prediction tasks, and this is a very active area of research (Min et al, 2017;Pattanayak, 2017;Namin et al, 2018). All DL algorithms employ multiple neuron layers and numerous architectures have been proposed: multiple layer perceptrons (MLPs), recurrent neural networks (RNNs), convolutional neural networks (CNNs) (LeCun et al, 2015) and others.…”

A Survey on Intelligent Agricultural Information Handling Methodologies

“…Namin et al [54] present a neural architecture consisting of a CNN and Long-Short Term Network (LSTN) for the task of plant classification. In particular, the authors utilize the CNN for the unsupervised generation of features, and the LSTN to investigate the growth of plants.…”