Canopy-attention-YOLOv4-based immature/mature apple fruit detection on dense-foliage tree architectures for early crop load estimation

Lu, Shenglian; Chen, Wenkang; Zhang, Xin; Karkee, Manoj

doi:10.1016/j.compag.2022.106696

Cited by 59 publications

(30 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Discussion. The effectiveness of variants of general object detection and small object detection for small apple detection has been widely used in a series of studies [31][32][33][34][35][36]. In this article, a balanced feature pyramid network achieved the precision improvement of small apple detection.…”

Section: Discussionmentioning

confidence: 99%

“…With the emergence of the attention mechanism and Transformer [ 34 ], the self-attention mechanism brings new inspirations to fruit detection. To detect immature/mature apples, the canopy-attention-YOLOv4 [ 35 ] was designed by introducing a convolutional block attention module to the generic YOLOv4 detector. A green pepper detection method [ 36 ] was proposed based on YOLOv4-tiny, which incorporates an attention mechanism and the idea of multiscale prediction.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

BFP Net: Balanced Feature Pyramid Network for Small Apple Detection in Complex Orchard Environment

Sun

Chen³

et al. 2022

Plant Phenomics

View full text Add to dashboard Cite

Despite of significant achievements made in the detection of target fruits, small fruit detection remains a great challenge, especially for immature small green fruits with a few pixels. The closeness of color between the fruit skin and the background greatly increases the difficulty of locating small target fruits in the natural orchard environment. In this paper, we propose a balanced feature pyramid network (BFP Net) for small apple detection. This network can balance information mapped to small apples from two perspectives: multiple-scale fruits on the different layers of FPN and a characteristic of a new extended feature from the output of ResNet50 conv1. Specifically, we design a weight-like feature fusion architecture on the lateral connection and top-down structure to alleviate the small-scale information imbalance on the different layers of FPN. Moreover, a new extended layer from ResNet50 conv1 is embedded into the lowest layer of standard FPN, and a decoupled-aggregated module is devised on this new extended layer of FPN to complement spatial location information and relieve the problem of locating small apple. In addition, a feature Kullback-Leibler distillation loss is introduced to transfer favorable knowledge from the teacher model to the student model. Experimental results show that APS of our method reaches 47.0%, 42.2%, and 35.6% on the benchmark of the GreenApple, MinneApple, and Pascal VOC, respectively. Overall, our method is not only slightly better than some state-of-the-art methods but also has a good generalization performance.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

BFP Net: Balanced Feature Pyramid Network for Small Apple Detection in Complex Orchard Environment

Sun

Chen³

et al. 2022

Plant Phenomics

View full text Add to dashboard Cite

show abstract

“…X. Li et al proposed a recognition method for green peppers by combining an attention module with an adaptive spatial feature pyramid [ 20 ]. Lu et al added the convolutional block attention module to the generic YOLOv4, improving the detection accuracy of mature apples by focusing on the target canopies [ 21 ]. Fu et al developed one kiwifruit detection network by adding two convolutional kernels of 3 × 3 and 1 × 1 to the fifth and sixth convolution layers of the YOLOv3-Tiny [ 22 ].…”

Section: Related Workmentioning

confidence: 99%

YOLOv5s-FP: A Novel Method for In-Field Pear Detection Using a Transformer Encoder and Multi-Scale Collaboration Perception

Rao

Jin

et al. 2022

Sensors

View full text Add to dashboard Cite

Precise pear detection and recognition is an essential step toward modernizing orchard management. However, due to the ubiquitous occlusion in orchards and various locations of image acquisition, the pears in the acquired images may be quite small and occluded, causing high false detection and object loss rate. In this paper, a multi-scale collaborative perception network YOLOv5s-FP (Fusion and Perception) was proposed for pear detection, which coupled local and global features. Specifically, a pear dataset with a high proportion of small and occluded pears was proposed, comprising 3680 images acquired with cameras mounted on a ground tripod and a UAV platform. The cross-stage partial (CSP) module was optimized to extract global features through a transformer encoder, which was then fused with local features by an attentional feature fusion mechanism. Subsequently, a modified path aggregation network oriented to collaboration perception of multi-scale features was proposed by incorporating a transformer encoder, the optimized CSP, and new skip connections. The quantitative results of utilizing the YOLOv5s-FP for pear detection were compared with other typical object detection networks of the YOLO series, recording the highest average precision of 96.12% with less detection time and computational cost. In qualitative experiments, the proposed network achieved superior visual performance with stronger robustness to the changes in occlusion and illumination conditions, particularly providing the ability to detect pears with different sizes in highly dense, overlapping environments and non-normal illumination areas. Therefore, the proposed YOLOv5s-FP network was practicable for detecting in-field pears in a real-time and accurate way, which could be an advantageous component of the technology for monitoring pear growth status and implementing automated harvesting in unmanned orchards.

show abstract

“…The test achieved an AP of 97.2%. Lu et al (2022) used the improved YOLOv4 to calculate the number and the size of fruits on the whole apple tree. The network had the highest detection rate during fruit picking.…”

Section: Introductionmentioning

confidence: 99%

YOLO-P: An efficient method for pear fast detection in complex orchard picking environment

Sun

Wang²,

Jing³

2023

Front. Plant Sci.

View full text Add to dashboard Cite

IntroductionFruit detection is one of the key functions of an automatic picking robot, but fruit detection accuracy is seriously decreased when fruits are against a disordered background and in the shade of other objects, as is commmon in a complex orchard environment.MethodsHere, an effective mode based on YOLOv5, namely YOLO-P, was proposed to detect pears quickly and accurately. Shuffle block was used to replace the Conv, Batch Norm, SiLU (CBS) structure of the second and third stages in the YOLOv5 backbone, while the inverted shuffle block was designed to replace the fourth stage’s CBS structure. The new backbone could extract features of pears from a long distance more efficiently. A convolutional block attention module (CBAM) was inserted into the reconstructed backbone to improve the robot’s ability to capture pears’ key features. Hard-Swish was used to replace the activation functions in other CBS structures in the whole YOLOv5 network. A weighted confidence loss function was designed to enhance the detection effect of small targets.ResultAt last, model comparison experiments, ablation experiments, and daytime and nighttime pear detection experiments were carried out. In the model comparison experiments, the detection effect of YOLO-P was better than other lightweight networks. The results showed that the module’s average precision (AP) was 97.6%, which was 1.8% higher than the precision of the original YOLOv5s. The model volume had been compressed by 39.4%, from 13.7MB to only 8.3MB. Ablation experiments verified the effectiveness of the proposed method. In the daytime and nighttime pear detection experiments, an embedded industrial computer was used to test the performance of YOLO-P against backgrounds of different complexities and when fruits are in different degrees of shade.DiscussionThe results showed that YOLO-P achieved the highest F1 score (96.1%) and frames per second (FPS) (32 FPS). It was sufficient for the picking robot to quickly and accurately detect pears in orchards. The proposed method can quickly and accurately detect pears in unstructured environments. YOLO-P provides support for automated pear picking and can be a reference for other types of fruit detection in similar environments.

show abstract

Canopy-attention-YOLOv4-based immature/mature apple fruit detection on dense-foliage tree architectures for early crop load estimation

Cited by 59 publications

References 33 publications

BFP Net: Balanced Feature Pyramid Network for Small Apple Detection in Complex Orchard Environment

BFP Net: Balanced Feature Pyramid Network for Small Apple Detection in Complex Orchard Environment

YOLOv5s-FP: A Novel Method for In-Field Pear Detection Using a Transformer Encoder and Multi-Scale Collaboration Perception

YOLO-P: An efficient method for pear fast detection in complex orchard picking environment

Contact Info

Product

Resources

About