Real-time detection and localization using SSD method for oyster mushroom picking robot

Yang, Qian; Rong, Jiacheng; Wang, Pengbo; Yang, Zhan; Geng, Changxing

doi:10.1109/rcar49640.2020.9303258

Cited by 15 publications

(6 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(d) SSD SSD is also a one-stage network; unlike the YOLO series, the SSD network has different scales and aspect ratios of Prior boxes, which allows for the use of different sizes of feature maps for the detection of targets of various scales. Qian et al [87] proposed an SSD-based method for accurate and real-time mushroom detection and location and optimized the backbone network in the original SSD model to improve the real-time detection performance in the embedded device. The model performs well in tests, with an F1 score of 0.951 and an average localization error of 2.43 mm for mushrooms.…”

Section: Perception Methods Based On Deep Learningmentioning

confidence: 99%

A Comprehensive Review of the Research of the “Eye–Brain–Hand” Harvesting System in Smart Agriculture

Ji,

Huang,

Wang

et al. 2023

Agronomy

View full text Add to dashboard Cite

Smart agricultural harvesting robots’ vision recognition, control decision, and mechanical hand modules all resemble the human eye, brain, and hand, respectively. To enable automatic and precise picking of target fruits and vegetables, the system makes use of cutting-edge sensor technology, machine vision algorithms, and intelligent control and decision methods. This paper provides a comprehensive review of international research advancements in the “eye–brain–hand” harvesting systems within the context of smart agriculture, encompassing aspects of mechanical hand devices, visual recognition systems, and intelligent decision systems. Then, the key technologies used in the current research are reviewed, including image processing, object detection and tracking, machine learning, deep learning, etc. In addition, this paper explores the application of the system to different crops and environmental conditions and analyzes its advantages and challenges. Finally, the challenges and prospects for the research on picking robots in the future are presented, including further optimization of the algorithm and improvement of flexibility and reliability of mechanical devices. To sum up, the “eye–brain–hand” picking system in intelligent agriculture has great potential to improve the efficiency and quality of crop picking and reduce labor pressure, and it is expected to be widely used in agricultural production.

show abstract

Section: Perception Methods Based On Deep Learningmentioning

confidence: 99%

A Comprehensive Review of the Research of the “Eye–Brain–Hand” Harvesting System in Smart Agriculture

Ji,

Huang,

Wang

et al. 2023

Agronomy

View full text Add to dashboard Cite

show abstract

“…Thus far, they are grown and harvested manually with little process automation; i.e., only the indoor growing environment is controlled and managed using a sensor-actuator setup for variables, such as temperature, humidity, and air quality [8,9]. Some rudimentary (semi-)automated harvesting mechanisms for mushrooms exist, e.g., [10][11][12], but they do not recognize the mushroom growth status and optimal harvest time. They do not consider mushroom damage caused by the harvesting process.…”

Section: Related Workmentioning

confidence: 99%

MushR: A Smart, Automated, and Scalable Indoor Harvesting System for Gourmet Mushrooms

Sujatanagarjuna,

Kia,

Briechle

et al. 2023

Agriculture

View full text Add to dashboard Cite

Gourmet mushrooms are foraged from the wild or grown indoors in controlled environments. Indoor mushroom farms with controlled growth environments allow for all-year-round growing. However, it remains a labor-intensive process. We propose MushR as a modular and scalable gourmet mushroom growing and harvesting system that goes beyond the state of the art, which merely monitors and controls the growing environment, by introducing an image recognition system that determines when and which mushrooms are ready to be harvested in conjunction with a proof of concept of an automated mushroom harvesting mechanism for harvesting the mushrooms without human interaction. The image recognition setup monitors the growing status of the mushrooms and guides the harvesting process. We present a Mask R-CNN model for the detection of oyster mushroom maturity with a 91.7% training accuracy and a semiautomated harvesting system, integrating a Raspberry Pi for control, an electrical switch, an air compressor, and a pneumatic cylinder with a cutting knife to facilitate timely mushroom harvesting. The modularity and scalability of the system allow for industry-level usage and can be scaled according to the required mushroom-growing systems within the facility. The AI model, its underlying dataset, a digital twin for mushroom production, the setup of our growth and control chambers, and additional information are all made available under an open-source license.

show abstract

“…As far as we know, research on mushrooms’ 3D posture estimation is limited. Qian et al [ 24 ] proposed an object detection and localization approach for an oyster mushroom-picking robot that combines detection information from a neural network with depth information from an RGB-D camera. This approach used the SSD object detection algorithm and depth images, based on binocular and structured light principles, to locate the precise position of the detected object in the 3D environment, guaranteeing real-time performance.…”

Section: Related Workmentioning

confidence: 99%

Mushroom Detection and Three Dimensional Pose Estimation from Multi-View Point Clouds

Retsinas

Efthymiou

Anagnostopoulou

et al. 2023

Sensors

View full text Add to dashboard Cite

Agricultural robotics is an up and coming field which deals with the development of robotic systems able to tackle a multitude of agricultural tasks efficiently. The case of interest, in this work, is mushroom collection in industrial mushroom farms. Developing such a robot, able to select and out-root a mushroom, requires delicate actions that can only be conducted if a well-performing perception module exists. Specifically, one should accurately detect the 3D pose of a mushroom in order to facilitate the smooth operation of the robotic system. In this work, we develop a vision module for 3D pose estimation of mushrooms from multi-view point clouds using multiple RealSense active–stereo cameras. The main challenge is the lack of annotation data, since 3D annotation is practically infeasible on a large scale. To address this, we developed a novel pipeline for mushroom instance segmentation and template matching, where a 3D model of a mushroom is the only data available. We evaluated, quantitatively, our approach over a synthetic dataset of mushroom scenes, and we, further, validated, qualitatively, the effectiveness of our method over a set of real data, collected by different vision settings.

show abstract

Real-time detection and localization using SSD method for oyster mushroom picking robot

Cited by 15 publications

References 10 publications

A Comprehensive Review of the Research of the “Eye–Brain–Hand” Harvesting System in Smart Agriculture

A Comprehensive Review of the Research of the “Eye–Brain–Hand” Harvesting System in Smart Agriculture

MushR: A Smart, Automated, and Scalable Indoor Harvesting System for Gourmet Mushrooms

Mushroom Detection and Three Dimensional Pose Estimation from Multi-View Point Clouds

Contact Info

Product

Resources

About