TSCMDL: Multimodal Deep Learning Framework for Classifying Tree Species Using Fusion of 2-D and 3-D Features

Liu, Bingjie; Hao, Yuanshuo; Huang, Huaguo; Chen, Shuxin; Li, Zengyuan; Chen, Erxue; Tian, Xin; Ren, Min

doi:10.1109/tgrs.2023.3266057

Cited by 4 publications

(7 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, resampling can lead to pixel blending, making it difficult for the model to distinguish between tree species with similar colors or textures. Chen et al (2023) [49] observed that, following the resampling of UAV imagery, the results of classifying vegetation species and ground objects using different machine learning classifiers decreased as the spatial resolution decreased. Thus, some misclassification of Japanese oak might be improved by using the UAV datasets at the original resolution.…”

Section: General Discussion Of the Misclassificationsmentioning

confidence: 99%

“…The traditional method of acquiring this information is based on individual tree species and thus time-consuming and laborious [5,6], as well as difficult to apply on complex uneven-mixed forests [7]. The recent development of remote sensing techniques has provided an effective and efficient means of obtaining highly accurate forest information [6].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Mapping the Distribution of High-Value Broadleaf Tree Crowns through Unmanned Aerial Vehicle Image Analysis Using Deep Learning

Htun,

Owari,

Tsuyuki

et al. 2024

Algorithms

View full text Add to dashboard Cite

High-value timber species with economic and ecological importance are usually distributed at very low densities, such that accurate knowledge of the location of these trees within a forest is critical for forest management practices. Recent technological developments integrating unmanned aerial vehicle (UAV) imagery and deep learning provide an efficient method for mapping forest attributes. In this study, we explored the applicability of high-resolution UAV imagery and a deep learning algorithm to predict the distribution of high-value deciduous broadleaf tree crowns of Japanese oak (Quercus crispula) in an uneven-aged mixed forest in Hokkaido, northern Japan. UAV images were collected in September and October 2022 before and after the color change of the leaves of Japanese oak to identify the optimal timing of UAV image collection. RGB information extracted from the UAV images was analyzed using a ResU-Net model (U-Net model with a Residual Network 101 (ResNet101), pre-trained on large ImageNet datasets, as backbone). Our results, confirmed using validation data, showed that reliable F1 scores (>0.80) could be obtained with both UAV datasets. According to the overlay analyses of the segmentation results and all the annotated ground truth data, the best performance was that of the model with the October UAV dataset (F1 score of 0.95). Our case study highlights a potential methodology to offer a transferable approach to the management of high-value timber species in other regions.

show abstract

Section: General Discussion Of the Misclassificationsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Mapping the Distribution of High-Value Broadleaf Tree Crowns through Unmanned Aerial Vehicle Image Analysis Using Deep Learning

Htun,

Owari,

Tsuyuki

et al. 2024

Algorithms

View full text Add to dashboard Cite

show abstract

“…Forests 2024, 15, x FOR PEER REVIEW 4 of 20 challenge in the process of multi-source remote sensing fusion [13]. Therefore, further indepth research is needed to investigate the application effects of different object detection models in complex forest stands, the precise registration between different data sources, and the impact of different data fusion methods and spatial resolutions on the performance of tree species identification models [49].…”

Section: Study Areamentioning

confidence: 99%

“…Due to the obvious advantages and disadvantages of RGB images and LiDAR data, some researchers have attempted to combine these two types of data for tree species identification [13][14][15][16]. In previous studies on the identification of individual tree species combining the two types of data, there are basically two steps: individual tree segmentation and tree species identification [17,18].…”

Section: Introductionmentioning

confidence: 99%

“…Generally, multi-source remote sensing data is typically acquired through different flight missions. Since the attitude and positioning of UAVs in these missions cannot always be completely consistent, leading to errors, precise data registration has become a significant challenge in the process of multi-source remote sensing fusion [13]. Therefore, further in-depth research is needed to investigate the application effects of different object detection models in complex forest stands, the precise registration between different data sources, and the impact of different data fusion methods and spatial resolutions on the performance of tree species identification models [49].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Individual Tree Species Identification for Complex Coniferous and Broad-Leaved Mixed Forests Based on Deep Learning Combined with UAV LiDAR Data and RGB Images

Zhong,

Zhang,

Liu

et al. 2024

Forests

View full text Add to dashboard Cite

Automatic and accurate individual tree species identification is essential for the realization of smart forestry. Although existing studies have used unmanned aerial vehicle (UAV) remote sensing data for individual tree species identification, the effects of different spatial resolutions and combining multi-source remote sensing data for automatic individual tree species identification using deep learning methods still require further exploration, especially in complex forest conditions. Therefore, this study proposed an improved YOLOv8 model for individual tree species identification using multisource remote sensing data under complex forest stand conditions. Firstly, the RGB and LiDAR data of natural coniferous and broad-leaved mixed forests under complex conditions in Northeast China were acquired via a UAV. Then, different spatial resolutions, scales, and band combinations of multisource remote sensing data were explored, based on the YOLOv8 model for tree species identification. Subsequently, the Attention Multi-level Fusion (AMF) Gather-and-Distribute (GD) YOLOv8 model was proposed, according to the characteristics of the multisource remote sensing forest data, in which the two branches of the AMF Net backbone were able to extract and fuse features from multisource remote sensing data sources separately. Meanwhile, the GD mechanism was introduced into the neck of the model, in order to fully utilize the extracted features of the main trunk and complete the identification of eight individual tree species in the study area. The results showed that the YOLOv8x model based on RGB images combined with current mainstream object detection algorithms achieved the highest mAP of 75.3%. When the spatial resolution was within 8 cm, the accuracy of individual tree species identification exhibited only a slight variation. However, the accuracy decreased significantly with the decrease of spatial resolution when the resolution was greater than 15 cm. The identification results of different YOLOv8 scales showed that x, l, and m scales could exhibit higher accuracy compared with other scales. The DGB and PCA-D band combinations were superior to other band combinations for individual tree identification, with mAP of 75.5% and 76.2%, respectively. The proposed AMF GD YOLOv8 model had a more significant improvement in tree species identification accuracy than a single remote sensing sources and band combinations data, with a mAP of 81.0%. The study results clarified the impact of spatial resolution on individual tree species identification and demonstrated the excellent performance of the proposed AMF GD YOLOv8 model in individual tree species identification, which provides a new solution and technical reference for forestry resource investigation combined multisource remote sensing data.

show abstract

A Review: Tree Species Classification Based on Remote Sensing Data and Classic Deep Learning-based Methods

Zhong,

Dai,

Fang

et al. 2024

Preprint

View full text Add to dashboard Cite

Timely and accurate information on tree species is crucial for the sustainable management of natural resources, forest inventory, biodiversity detection, and carbon stock calculation. The advancement of remote sensing technology and artificial intelligence has facilitated the acquisition and analysis of remote sensing data, resulting in more precise and effective classification of tree species. Multimodal remote sensing data and deep learning seem to be the current tree species classification research mainstream, whether or not. The current review on the remote sensing data and deep learning tree species classification methods perspectives to analyze the unimodal and multimodal remote sensing data and classification methods in this realm is missing. To bridge the gap, we search for major trends in the remote sensing data and tree species classification methods, provide a detailed overview of classic deep learning-based methods for tree species classification, and discuss the limitations.

show abstract

TSCMDL: Multimodal Deep Learning Framework for Classifying Tree Species Using Fusion of 2-D and 3-D Features

Cited by 4 publications

References 51 publications

Mapping the Distribution of High-Value Broadleaf Tree Crowns through Unmanned Aerial Vehicle Image Analysis Using Deep Learning

Mapping the Distribution of High-Value Broadleaf Tree Crowns through Unmanned Aerial Vehicle Image Analysis Using Deep Learning

Individual Tree Species Identification for Complex Coniferous and Broad-Leaved Mixed Forests Based on Deep Learning Combined with UAV LiDAR Data and RGB Images

A Review: Tree Species Classification Based on Remote Sensing Data and Classic Deep Learning-based Methods

Contact Info

Product

Resources

About