ACE R-CNN: An Attention Complementary and Edge Detection-Based Instance Segmentation Algorithm for Individual Tree Species Identification Using UAV RGB Images and LiDAR Data

Abstract

Accurate and automatic identification of tree species information at the individual tree scale is of great significance for fine-scale investigation and management of forest resources and scientific assessment of forest ecosystems. Despite the fact that numerous studies have been conducted on the delineation of individual tree crown and species classification using drone high-resolution red, green and blue (RGB) images, and Light Detection and Ranging (LiDAR) data, performing the above tasks simultaneously has rarely been explored, especially in complex forest environments. In this study, we improve upon the state of the Mask region-based convolution neural network (Mask R-CNN) with our proposed attention complementary network (ACNet) and edge detection R-CNN (ACE R-CNN) for individual tree species identification in high-density and complex forest environments. First, we propose ACNet as the feature extraction backbone network to fuse the weighted features extracted from RGB images and canopy height model (CHM) data through an attention complementary module, which is able to selectively fuse weighted features extracted from RGB and CHM data at different scales, and enables the network to focus on more effective information. Second, edge loss is added to the loss function to improve the edge accuracy of the segmentation, which is calculated through the edge detection filter introduced in the Mask branch of Mask R-CNN. We demonstrate the performance of ACE R-CNN for individual tree species identification in three experimental areas of different tree species in southern China with precision (P), recall (R), F1-score, and average precision (AP) above 0.9. Our proposed ACNet–the backbone network for feature extraction–has better performance in individual tree species identification compared with the ResNet50-FPN (feature pyramid network). The addition of the edge loss obtained by the Sobel filter further improves the identification accuracy of individual tree species and accelerates the convergence speed of the model training. This work demonstrates the improved performance of ACE R-CNN for individual tree species identification and provides a new solution for tree-level species identification in complex forest environments, which can support carbon stock estimation and biodiversity assessment.

Fine Classification of Urban Tree Species Based on UAV-Based RGB Imagery and LiDAR Data

“…However, the fused RGB and LiDAR features have increased the overall accuracy by 18.49% which is consistent with the results of many studies [17,59]. For example, Li et al [38] have used an improved algorithm to conduct individual tree species identification based on UAV RGB imagery and LiDAR data. And the results have proved that the combination of RGB and LiDAR data is much more suitable for tree species classification.…”

“…studies that integrated UAV-based LiDAR and RGB imagery for tree species classification [36][37][38].…”

“…However, previous studies related to tree species classification mainly involved multispectral data from satellites or airborne platforms [34,35]. There have been limited studies that integrated UAV-based LiDAR and RGB imagery for tree species classification [36][37][38].…”

Tree-Gpt: Modular Large Language Model Expert System for Forest Remote Sensing Image Understanding and Interactive Analysis

“…Some methods have improved edge extraction by enhancing the loss function (Zhang et al, 2022), while others have considered incorporating multimodal feature extraction techniques within the Mask-RCNN framework. They simultaneously use texture, color, and elevation information from both RGB and CHM images to improve tree segmentation accuracy (Li et al, 2022). Although deep learning-based instance segmentation methods for singletree segmentation show significant improvements over traditional image segmentation methods in terms of robustness and generalization, most supervised deep learning methods suffer from a fundamental limitation: their performance is heavily constrained by the training data.…”

ExtSpecR : An R Package and Tool for Extracting Tree Spectra from UAV-Based Remote Sensing

“…R-CNN models analyze the image and extract self-features based on a predefined ground truth of each tree in the image. Therefore, CNN-based approaches show better performance for tree detection and classification than traditional segmentation algorithms [ 19 ]. However, it should be noted that these approaches require manual labeling (annotation) to generate the ground truth for each tree in the image, and the accuracy of the annotation process affects the overall accuracy of the models used to train the R-CNN model.…”