A workflow for segmenting soil and plant X-ray computed tomography images with deep learning in Google’s Colaboratory

Abstract

X-ray micro-computed tomography (X-ray μCT) has enabled the characterization of the properties and processes that take place in plants and soils at the micron scale. Despite the widespread use of this advanced technique, major limitations in both hardware and software limit the speed and accuracy of image processing and data analysis. Recent advances in machine learning, specifically the application of convolutional neural networks to image analysis, have enabled rapid and accurate segmentation of image data. Yet, challenges remain in applying convolutional neural networks to the analysis of environmentally and agriculturally relevant images. Specifically, there is a disconnect between the computer scientists and engineers, who build these AI/ML tools, and the potential end users in agricultural research, who may be unsure of how to apply these tools in their work. Additionally, the computing resources required for training and applying deep learning models are unique, more common to computer gaming systems or graphics design work, than to traditional computational systems. To navigate these challenges, we developed a modular workflow for applying convolutional neural networks to X-ray μCT images, using low-cost resources in Google’s Colaboratory web application. Here we present the results of the workflow, illustrating how parameters can be optimized to achieve best results using example scans from walnut leaves, almond flower buds, and a soil aggregate. We expect that this framework will accelerate the adoption and use of emerging deep learning techniques within the plant and soil sciences.

Segmentation of plant residues on soil X‐ray CT images using neural network

“…We can compare our results with Rippner et al. (2022), despite the fact that they used alternative segmentation quality metric, the F1 score. For this reason, IOU was recalculated to the F1 score based on the methodology of Olczak et al.…”

“…In soil, a similar approach was implemented in the multiclass segmentation of microtomographic images of soil aggregates with particulate organic matter (POM; Rippner et al, 2022). We can compare our results with Rippner et al (2022), despite the fact that they used alternative segmentation quality metric, the F1 score. For this reason, IOU was recalculated to the F1 score based on the methodology of Olczak et al (2021).…”

HopBox: An image analysis pipeline to characterize hop cone morphology

“…Models were trained on two image feature classes, hops and background. Model performance was evaluated based on accuracy (the percentage of all correctly classified observations) and F1 score (the harmonic mean of r calculated from evaluating the six image/annotation pairs that were not included in the training or validation datasets) (Rippner et al., 2022). In this study, accuracy is defined as the percentage of all correctly classified pixels, while the F1 score is defined as the harmonic mean of precision (the % of correct positive predictions out of all positive predictions) and recall (the % of correct positive predictions out of all possible positive predictions).…”

“…Ten models were trained for 250 epochs using a binary cross‐entropy loss function. An Adam optimizer for stochastic optimization with the learning rate set to 0.001 of the image/annotation pairs was used for training and 20% were used for the validation of the model during training (Rippner et al., 2022). The batch size was set to 1, and images were scaled to 0.5 size in both the x and y dimensions for training due to graphics processing unit VRAM limitations when working with large images (5184 × 3456 pixels).…”

“…A Python-based PyTorch implementation of a fully convolutional network with a ResNet-101 backbone was trained using 12 pairs of images and annotations (He et al, 2015;Long et al, 2015;Oliphant, 2007;Paszke et al, 2019;Rippner et al, 2022). Annotated images were created in ImageJ (Schneider et al, 2012) by creating binary black and white images to designate hops and background.…”