Real-time guava tree-part segmentation using fully convolutional network with channel and spatial attention

Abstract

It is imminent to develop intelligent harvesting robots to alleviate the burden of rising costs of manual picking. A key problem in robotic harvesting is how to recognize tree parts efficiently without losing accuracy, thus helping the robots plan collision-free paths. This study introduces a real-time tree-part segmentation network by improving fully convolutional network with channel and spatial attention. A lightweight backbone is first deployed to extract low-level and high-level features. These features may contain redundant information in their channel and spatial dimensions, so a channel and spatial attention module is proposed to enhance informative channels and spatial locations. On this basis, a feature aggregation module is investigated to fuse the low-level details and high-level semantics to improve segmentation accuracy. A tree-part dataset with 891 RGB images is collected, and each image is manually annotated in a per-pixel fashion. Experiment results show that when using MobileNetV3-Large as the backbone, the proposed network obtained an intersection-over-union (IoU) value of 63.33 and 66.25% for the branches and fruits, respectively, and required only 2.36 billion floating point operations per second (FLOPs); when using MobileNetV3-Small as the backbone, the network achieved an IoU value of 60.62 and 61.05% for the branches and fruits, respectively, at a speed of 1.18 billion FLOPs. Such results demonstrate that the proposed network can segment the tree-parts efficiently without loss of accuracy, and thus can be applied to the harvesting robots to plan collision-free paths.

A Review on Recent Deep Learning-Based Semantic Segmentation for Urban Greenness Measurement

“…In the greenness measurement, green objects, including trees, shrubs, grass, and related objects, could be treated as "things", and the others, including "sky", "car", "road", "sidewalk", "building", etc., could be stuff. The most recent technology tries to divide the "things" into "parts" and to recognize them separately (i.e., the crown and trunk as parts of a tree) [63].…”

“…The most recent technology tries to divide the "things" into "parts" and to recognize them separately (i.e., the crown and trunk as parts of a tree) [63]. Thus, we can say that the recent DL-based segmentation technology could provide a new horizon in landscape analysis by elaborate segmentation techniques.…”

Deep Learning-Based Segmentation of Intertwined Fruit Trees for Agricultural Tasks

“…Tree segmentation was not performed. Lin proposed deep learning models to segment branches and fruits for the path planning of a harvesting robot [20]. The authors modified MobileNetV3 to develop an attention mechanism.…”

Testing a Method Based on an Improved UNet and Skeleton Thinning Algorithm to Obtain Branch Phenotypes of Tall and Valuable Trees Using Abies beshanzuensis as the Research Sample

“…Compared with the original PSPNet, the intersection over union (IoU) value of the improved model increased by 0.67%. Lin et al [20] proposed a real-time tree segmentation method with a branch IoU of 63.33% via an improved FCN. In the above study, the deep learning algorithm was successfully applied to various plant detection and segmentation applications, effectively addressing the limitations of the traditional machine vision methods.…”