Local Motion Planner for Autonomous Navigation in Vineyards with a RGB-D Camera-Based Algorithm and Deep Learning Synergy

Abstract

With the advent of agriculture 3.0 and 4.0, in view of efficient and sustainable use of resources, researchers are increasingly focusing on the development of innovative smart farming and precision agriculture technologies by introducing automation and robotics into the agricultural processes. Autonomous agricultural field machines have been gaining significant attention from farmers and industries to reduce costs, human workload, and required resources. Nevertheless, achieving sufficient autonomous navigation capabilities requires the simultaneous cooperation of different processes; localization, mapping, and path planning are just some of the steps that aim at providing to the machine the right set of skills to operate in semi-structured and unstructured environments. In this context, this study presents a low-cost, power-efficient local motion planner for autonomous navigation in vineyards based only on an RGB-D camera, low range hardware, and a dual layer control algorithm. The first algorithm makes use of the disparity map and its depth representation to generate a proportional control for the robotic platform. Concurrently, a second back-up algorithm, based on representations learning and resilient to illumination variations, can take control of the machine in case of a momentaneous failure of the first block generating high-level motion primitives. Moreover, due to the double nature of the system, after initial training of the deep learning model with an initial dataset, the strict synergy between the two algorithms opens the possibility of exploiting new automatically labeled data, coming from the field, to extend the existing model’s knowledge. The machine learning algorithm has been trained and tested, using transfer learning, with acquired images during different field surveys in the North region of Italy and then optimized for on-device inference with model pruning and quantization. Finally, the overall system has been validated with a customized robot platform in the appropriate environment.

Learned Visual Navigation for Under-Canopy Agricultural Robots

“…This has motivated vision-based navigation systems. Past work in vision-based agricultural navigation can be classified into over the canopy [72,26,69,34,4], under-canopy in orchards [59,51,7,1] and under-canopy in row crops and horticultural crops [67,27]. Vanishing lines based heuristics was commonly used in these works.…”

“…Incidentally, corn and soybean acerage is atleast 10× larger than orchards. Recent visual servoing with RGB-D has been used for orchar navigation [1], however this approach will not work in corn-soybean canopies due to visual clutter and small-size of crops earlier in the growing season. Classical Navigation.…”

Obstacle Detection System for Agricultural Mobile Robot Application Using RGB-D Cameras

“…Movement, navigation, and obstacle avoidance are crucial aspects of all robotic systems, irrespective of robot design, application field, and performed tasks (e.g., [ 15 , 16 , 17 ].) While most agricultural robots move on fields and navigate their path along crop or orchard rows [ 3 , 8 , 15 , 18 ], lawn mowers traverse soft surfaces, require high maneuverability, and navigate through an area which is quite uniform with no natural landmarks to follow. This makes it more challenging to navigate the robot, detect deviations from the desired path, and complicate obstacle detection.…”

Grape Berry Detection and Size Measurement Based on Edge Image Processing and Geometric Morphology

“…In recent years, there are mainly two methods based on 3D information and on 2D images in the use of machine vision to estimate the number of grape berries [6,7]. Huerta [8] and Rist [9] used 3D point cloud equipment to scan the 3D information of grape spikes to obtain parameters such as the geometry and structure of the grapes to reconstruct the grape phenotype to estimate the number of grape berries [10].…”