Application of consumer RGB-D cameras for fruit detection and localization in field: A critical review

Fu, Longsheng; Gao, Feng; Wu, Jingzhu; Li, Rui; Karkee, Manoj; Zhang, Qin

doi:10.1016/j.compag.2020.105687

Cited by 205 publications

(80 citation statements)

References 81 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Meanwhile, the cost of computer hardware has been greatly reduced, and the computing power of GPU has been remarkably improved. Deep learning has been extended to the agricultural field [ 118 , 119 , 120 ]. Methods based on deep learning have achieved good results in weed detection and classification [ 121 ].…”

Section: Weed Detection and Identification Methods Based On Deep Learningmentioning

confidence: 99%

Review of Weed Detection Methods Based on Computer Vision

Chen

Zhao

et al. 2021

Sensors

156

View full text Add to dashboard Cite

Weeds are one of the most important factors affecting agricultural production. The waste and pollution of farmland ecological environment caused by full-coverage chemical herbicide spraying are becoming increasingly evident. With the continuous improvement in the agricultural production level, accurately distinguishing crops from weeds and achieving precise spraying only for weeds are important. However, precise spraying depends on accurately identifying and locating weeds and crops. In recent years, some scholars have used various computer vision methods to achieve this purpose. This review elaborates the two aspects of using traditional image-processing methods and deep learning-based methods to solve weed detection problems. It provides an overview of various methods for weed detection in recent years, analyzes the advantages and disadvantages of existing methods, and introduces several related plant leaves, weed datasets, and weeding machinery. Lastly, the problems and difficulties of the existing weed detection methods are analyzed, and the development trend of future research is prospected.

show abstract

Section: Weed Detection and Identification Methods Based On Deep Learningmentioning

confidence: 99%

Review of Weed Detection Methods Based on Computer Vision

Chen

Zhao

et al. 2021

Sensors

156

View full text Add to dashboard Cite

show abstract

“…Artificial apple picking is a labor-intensive and time-intensive task. Therefore, in order to realize the efficient and automatic picking of apples, to ensure timely harvest of mature fruits, and improve the competitiveness of the apple market, further study of the key technologies of the apple picking robot is essential [1,2]. The intelligent perception and acquisition of apple information is one of the most critical technologies for the apple picking robot, which belongs to the information perception of the front-end part of the robot.…”

Section: Introductionmentioning

confidence: 99%

A Real-Time Apple Targets Detection Method for Picking Robot Based on Improved YOLOv5

et al. 2021

View full text Add to dashboard Cite

The apple target recognition algorithm is one of the core technologies of the apple picking robot. However, most of the existing apple detection algorithms cannot distinguish between the apples that are occluded by tree branches and occluded by other apples. The apples, grasping end-effector and mechanical picking arm of the robot are very likely to be damaged if the algorithm is directly applied to the picking robot. Based on this practical problem, in order to automatically recognize the graspable and ungraspable apples in an apple tree image, a light-weight apple targets detection method was proposed for picking robot using improved YOLOv5s. Firstly, BottleneckCSP module was improved designed to BottleneckCSP-2 module which was used to replace the BottleneckCSP module in backbone architecture of original YOLOv5s network. Secondly, SE module, which belonged to the visual attention mechanism network, was inserted to the proposed improved backbone network. Thirdly, the bonding fusion mode of feature maps, which were inputs to the target detection layer of medium size in the original YOLOv5s network, were improved. Finally, the initial anchor box size of the original network was improved. The experimental results indicated that the graspable apples, which were unoccluded or only occluded by tree leaves, and the ungraspable apples, which were occluded by tree branches or occluded by other fruits, could be identified effectively using the proposed improved network model in this study. Specifically, the recognition recall, precision, mAP and F1 were 91.48%, 83.83%, 86.75% and 87.49%, respectively. The average recognition time was 0.015 s per image. Contrasted with original YOLOv5s, YOLOv3, YOLOv4 and EfficientDet-D0 model, the mAP of the proposed improved YOLOv5s model increased by 5.05%, 14.95%, 4.74% and 6.75% respectively, the size of the model compressed by 9.29%, 94.6%, 94.8% and 15.3% respectively. The average recognition speeds per image of the proposed improved YOLOv5s model were 2.53, 1.13 and 3.53 times of EfficientDet-D0, YOLOv4 and YOLOv3 and model, respectively. The proposed method can provide technical support for the real-time accurate detection of multiple fruit targets for the apple picking robot.

show abstract

“…Concurrent with the development of various ground-based and aerial (e.g., unmanned aerial vehicle (UAV)) HTP systems is the rise in use of imaging sensors for phenotyping purposes. Sensors for colour (RGB), thermal, spectral (multi-and hyperspectral), and 3D (e.g., LiDAR) imaging have been applied extensively for phenotyping applications encompassing plant morphology, physiology, development, and postharvest quality [3][4][5][6]. Consequently, the meteoric rise in big image data arising from HTP systems necessitates the development of efficient image processing and analytical pipelines.…”

Section: Introductionmentioning

confidence: 99%

Automated Machine Learning for High-Throughput Image-Based Plant Phenotyping

2021

View full text Add to dashboard Cite

Automated machine learning (AutoML) has been heralded as the next wave in artificial intelligence with its promise to deliver high-performance end-to-end machine learning pipelines with minimal effort from the user. However, despite AutoML showing great promise for computer vision tasks, to the best of our knowledge, no study has used AutoML for image-based plant phenotyping. To address this gap in knowledge, we examined the application of AutoML for image-based plant phenotyping using wheat lodging assessment with unmanned aerial vehicle (UAV) imagery as an example. The performance of an open-source AutoML framework, AutoKeras, in image classification and regression tasks was compared to transfer learning using modern convolutional neural network (CNN) architectures. For image classification, which classified plot images as lodged or non-lodged, transfer learning with Xception and DenseNet-201 achieved the best classification accuracy of 93.2%, whereas AutoKeras had a 92.4% accuracy. For image regression, which predicted lodging scores from plot images, transfer learning with DenseNet-201 had the best performance (R2 = 0.8303, root mean-squared error (RMSE) = 9.55, mean absolute error (MAE) = 7.03, mean absolute percentage error (MAPE) = 12.54%), followed closely by AutoKeras (R2 = 0.8273, RMSE = 10.65, MAE = 8.24, MAPE = 13.87%). In both tasks, AutoKeras models had up to 40-fold faster inference times compared to the pretrained CNNs. AutoML has significant potential to enhance plant phenotyping capabilities applicable in crop breeding and precision agriculture.

show abstract

Application of consumer RGB-D cameras for fruit detection and localization in field: A critical review

Cited by 205 publications

References 81 publications

Review of Weed Detection Methods Based on Computer Vision

Review of Weed Detection Methods Based on Computer Vision

A Real-Time Apple Targets Detection Method for Picking Robot Based on Improved YOLOv5

Automated Machine Learning for High-Throughput Image-Based Plant Phenotyping

Contact Info

Product

Resources

About