Capturing long‐tailed individual tree diversity using an airborne imaging and a multi‐temporal hierarchical model

Abstract

Measuring forest biodiversity using terrestrial surveys is expensive and can only capture common species abundance in large heterogeneous landscapes. In contrast, combining airborne imagery with computer vision can generate individual tree data at the scales of hundreds of thousands of trees. To train computer vision models, ground‐based species labels are combined with airborne reflectance data. Due to the difficulty of finding rare species in a large landscape, many classification models only include the most abundant species, leading to biased predictions at broad scales. For example, if only common species are used to train the model, this assumes that these samples are representative across the entire landscape. Extending classification models to include rare species requires targeted data collection and algorithmic improvements to overcome large data imbalances between dominant and rare taxa. We use a targeted sampling workflow to the Ordway Swisher Biological Station within the US National Ecological Observatory Network (NEON), where traditional forestry plots had identified six canopy tree species with more than 10 individuals at the site. Combining iterative model development with rare species sampling, we extend a training dataset to include 14 species. Using a multi‐temporal hierarchical model, we demonstrate the ability to include species predicted at <1% frequency in landscape without losing performance on the dominant species. The final model has over 75% accuracy for 14 species with improved rare species classification compared to 61% accuracy of a baseline deep learning model. After filtering out dead trees, we generate landscape species maps of individual crowns for over 670 000 individual trees. We find distinct patches of forest composed of rarer species at the full‐site scale, highlighting the importance of capturing species diversity in training data. We estimate the relative abundance of 14 species within the landscape and provide three measures of uncertainty to generate a range of counts for each species. For example, we estimate that the dominant species, Pinus palustris accounts for c. 28% of predicted stems, with models predicting a range of counts between 160 000 and 210 000 individuals. These maps provide the first estimates of canopy tree diversity within a NEON site to include rare species and provide a blueprint for capturing tree diversity using airborne computer vision at broad scales.

Individual canopy tree species maps for the National Ecological Observatory Network

“…Following tree detection, we classified each predicted crown as “Alive” or “Dead” based on the RGB data. Presented in [ 27 ], this Alive-Dead model is a 2 class resnet-50 deep learning neural network trained on hand-annotated images from across all NEON sites. During prediction, the location of each predicted crown was cropped and passed to the Alive-Dead model for labeling as Alive (0) or Dead (1) with a confidence score for each class.…”

“…To classify each predicted crown tree crown to species, we use the 1-m hyperspectral data and a multi-temporal hierarchical model. Ref [ 27 ] found that a hierarchical model outperforms a flat model by improving rare species accuracy. The hierarchical model organizes tree species into submodels, allowing each model to learn better features related to distinguishing similar classes.…”

“…The changes in local species abundance over large areas contributes to further mismatch between training data and predicted landscapes at wide extents. Ref [ 27 ] proposed an approach to addressing these limitations by using a flexible hierarchical model structure that uses simple rules to define a series of models to create an ensemble species prediction. This approach uses both multiple views of the same crown across years, as well as a hierarchical structure to reduce the effect of species imbalance.…”

“…It was effective at expanding the number of species that could be accurately classified at a single National Ecological Observatory Network (NEON) site but has yet to be tested and applied across sites with a diversity of forest types. Here, we apply the tree delineation and species classification workflow proposed for a single site in [ 27 ] to sites across the United States and assess its performance in order to provide data for ecological and computer vision research.…”

Data science competition for cross-site individual tree species identification from airborne remote sensing data

“…‘Pine’, and were limited to less than 10 classes ( Persson et al, 2004 ; Heikkinen et al, 2010 ). The emergence of deep learning networks in computer vision, combined with greater data availability, led to a large number of publications combining deep learning with a variety of sensors and data acquisition platforms ( Fricker et al, 2019 ; Kattenborn et al, 2021 ; Mäyrä et al, 2021 ; Weinstein et al, 2023 ). A defining challenge of individual species classification is the fine-grained nature of the task, with subtle differences among co-occurring species, often within the same taxonomic genus.…”