A Dragon Fruit Picking Detection Method Based on YOLOv7 and PSP-Ellipse

Abstract

Dragon fruit is one of the most popular fruits in China and Southeast Asia. It, however, is mainly picked manually, imposing high labor intensity on farmers. The hard branches and complex postures of dragon fruit make it difficult to achieve automated picking. For picking dragon fruits with diverse postures, this paper proposes a new dragon fruit detection method, not only to identify and locate the dragon fruit, but also to detect the endpoints that are at the head and root of the dragon fruit, which can provide more visual information for the dragon fruit picking robot. First, YOLOv7 is used to locate and classify the dragon fruit. Then, we propose a PSP-Ellipse method to further detect the endpoints of the dragon fruit, including dragon fruit segmentation via PSPNet, endpoints positioning via an ellipse fitting algorithm and endpoints classification via ResNet. To test the proposed method, some experiments are conducted. In dragon fruit detection, the precision, recall and average precision of YOLOv7 are 0.844, 0.924 and 0.932, respectively. YOLOv7 also performs better compared with some other models. In dragon fruit segmentation, the segmentation performance of PSPNet on dragon fruit is better than some other commonly used semantic segmentation models, with the segmentation precision, recall and mean intersection over union being 0.959, 0.943 and 0.906, respectively. In endpoints detection, the distance error and angle error of endpoints positioning based on ellipse fitting are 39.8 pixels and 4.3°, and the classification accuracy of endpoints based on ResNet is 0.92. The proposed PSP-Ellipse method makes a great improvement compared with two kinds of keypoint regression method based on ResNet and UNet. Orchard picking experiments verified that the method proposed in this paper is effective. The detection method proposed in this paper not only promotes the progress of the automatic picking of dragon fruit, but it also provides a reference for other fruit detection.

An improved deep learning network for image detection and its application in Dendrobii caulis decoction piece

“…Compared to classic algorithms such as YOLOv3 39 and YOLOv4 40 , YOLOv5 41 boasts a more advanced network architecture that offers improved performance characteristics. In contrast to more recent and sophisticated models like YOLOv7 42 and YOLOv8 43 , it employs a more lightweight architectural design, enabling it to achieve the desired performance on our dataset with a significantly reduced computational footprint. Therefore, this paper proposes an improved YOLOv5 algorithm for the identification of D. caulis decoction piece.…”

“…This paper focus on the research involving the use of deep learning models to achieve high-accuracy detection or recognition of different plants or fruits.Zhou et al 42 used a PSPNet to detect the endpoints of the dragon fruit, including dragon fruit segmentation and position,achieved an accuracy of around 95%. On the other hand, Huang et al 52 designed a deep learning network that combines UAV data collection, AI embedded device, and target detection algorithm to detection citrus with an accuracy of 93.32%.…”

Optimizing the YOLOv7-Tiny Model with Multiple Strategies for Citrus Fruit Yield Estimation in Complex Scenarios

“…Cardellicchio et al [5] used the YOLOv5 detection network to study the phenotypic characteristics of tomato plants, so that tomato fruits and flowers could be accurately identified. In addition, the target detection networks have also demonstrated a good detection performance in fruit identification and counting [6][7][8][9][10]. Bi et al [11] used migration learning to train the FastR-CNN model for citrus fruit recognition with an average accuracy of 86.6%.…”

Designing and Testing a Picking and Selecting Integrated Remote-Operation-Type Dragon-Fruit-Picking Device

“…This greatly promotes the development of the domestic dragon fruit industry, as dragon fruit picking is an important part of the dragon fruit industry, as well as being an essential link [14]; however, because dragon fruit roots are very hard and the fruit growth stages are diverse, it is difficult to program a market fruit-picking machine to distinguish between the different fruit stages of dragon fruit picking. Therefore, it is difficult to realize the mechanized harvesting of dragon fruit [15], and, at present, dragon fruit is usually picked manually, which is labor-intensive and has low picking efficiency, restricting the development of the dragon fruit industry. In the main production areas of dragon fruit, owing to the large area and wide range of dragon fruit planting, manual picking cannot satisfy the requirements of efficient picking, so mechanized picking is an inevitable choice [16].…”