Deep-learning-based in-field citrus fruit detection and tracking

Abstract

Fruit yield estimation is crucial to establish fruit harvesting and marketing strategies. Recently, computer vision and deep learning techniques have been used to estimate citrus fruit yield and have exhibited a notable fruit detection ability. However, computer-vision-based citrus fruit counting has two key limitations: inconsistent fruit detection accuracy and double-counting of the same fruit. Using oranges as the experimental material, this paper proposes a deep-learning-based orange counting algorithm using video sequences to help overcome these problems. The algorithm consists of two sub-algorithms, OrangeYolo for fruit detection and OrangeSort for fruit tracking. The OrangeYolo backbone network is partially based on the YOLOv3 algorithm and improved upon to detect small object fruits at multiple scales. The network structure was adjusted to detect small-scale targets while enabling multiscale target detection. A channel attention and spatial attention multiscale fusion module was introduced to fuse the semantic features of the deep network with the shallow textural detail features. OrangeYolo can reach mean Average Precision (mAP) to 0.957 in the citrus dataset, which is higher than the 0.905, 0.911, and 0.917 that the YOLOv3, YOLOv4 and YOLOv5 algorithms. OrangeSort was designed to alleviate the double-counting problem of occluded fruits. A specific tracking region counting strategy and tracking algorithm based on motion displacement estimation are established. Six video sequences, which were taken from two fields containing 22 trees, were used as a validation dataset. The proposed method showed better performance (Mean Absolute Error(MAE) = 0.081, Standard Deviation(SD) = 0.08) compared to video-based manual counting and demonstrated more accurate results compared with existing standard Sort and DeepSort (MAE = 0.45, 1.212; SD = 0.4741, 1.3975; respectively).

EasyDAM_V2: Efficient Data Labeling Method for Multishape, Cross-Species Fruit Detection

“…First, we input the labeled source domain fruit dataset, translate the source domain fruit image to the target domain synthetic fruit image by the fruit image translation model (in the image generation module), and construct the labeled target domain synthetic fruit dataset (by combining the label data of the source domain fruit dataset). The fruit detection model OrangeYolo [ 36 ] proposed in our previous research work is then applied to construct the target domain pretrained fruit detection model. In the image translation module, based on the original CycleGAN, this paper proposes the Across-CycleGAN (introduced in Section 2.2 ).…”

“…Therefore, this paper proposes an adaptive threshold selection strategy for pseudolabel generation methods by combining the fruit detection model OrangeYolo [ 36 ] proposed in our previous research work. The strategy, based on the target domain pretrained fruit detection model constructed by Across-CycleGAN, calculates the quantity and score information characteristics of the pseudolabel generated under the corresponding confidence threshold conditions to obtain the quality variance values of pseudolabel.…”

A Tea Buds Counting Method Based on YOLOv5 and Kalman Filter Tracking Algorithm

“…Some new tracking strategies and CNN-based detection algorithms can solve this problem. For instance, the counting algorithm based on YOLO and correlation filtering still shows good results in complex environments [ 21 , 41 ].…”

Fruit Type Recognition Using Hybrid Method with Principal Component Analysis (PCA)

“…In order to detect the type of fruit, the system must be able to recognize it accurately. Many academics propose deep learning-based fruit recognition techniques to address the issue of fruit detection accuracy [1]. Fruit recognition can assist fruit vendors in identifying and distinguishing various fruit varieties that share some characteristics [2].…”