Robotic kiwifruit harvesting using machine vision, convolutional neural networks, and robotic arms

Abstract

As labor requirements in horticultural increase, so too does the feasibility of increased automation in these industries. This paper presents a performance evaluation of a kiwifruit harvesting robot designed to operate autonomously in pergola style orchards. The robot consists of four harvesting arms, endeffectors designed specifically for kiwifruit detachment, and a machine vision system employing convolution neural networks. Performance evaluations are presented for the harvester as a whole, as well as the machine vision system. We show the system as a whole is capable of harvesting over half of all fruit within three test orchards, equating a substantial reduction in peak harvesting labor requirements.

Improvements to and large‐scale evaluation of a robotic kiwifruit harvester

“…In 2017 our kiwifruit harvester was able to harvest 51.0% of 1,456 kiwifruit across three realistic commercial orchards with an average cycle‐time of 5.5 s/fruit (H. A. Williams et al, ). However to be commercially feasible, a kiwifruit harvester is required to harvest at least 80% of fruit with a cycle‐time of 0.25 s/fruit.…”

“…Detection of apples (Bargoti & Underwood, b; Dias et al, ; Inthiyaz, Kishore, & Madhav, ; Moallem, Serajoddin, & Pourghassem, ; Prasad et al, ; Puttemans, Vanbrabant, Tits, & Goedemé, ; Soleimani Pour, Chegini, Zarafshan, & Massah, ) and strawberries (Habaragamuwa et al, ; Puttemans et al, ) has shown good results with detection rates up to 90% of the fruit under real‐world orchard conditions. A kiwifruit detection system using semantic segmentation was able to detect 76.0% of kiwifruit in a real‐world orchard (H. A. Williams et al, ). Furthermore, a kiwifruit flower detection system using Faster‐RCNN was capable of detecting 79.8% of kiwifruit flowers, indicating its potential use for detecting kiwifruit under the same conditions (H. Williams et al, ).…”

“…Figure presents the harvesting system that was designed and built to meet these commercial requirements that were presented in H. A. Williams et al (). The harvesting system measures 2,410 mm × 1,130 mm × 370 mm and comprises four submodules evenly spaced 600 mm across the width.…”

“…A novel robotic kiwifruit harvester has been designed to help ease the growing requirement for seasonal workers (H. A. Williams et al, ). The work is a collaboration between the University of Auckland, the University of Waikato, Plant & Food Research, and Robotics Plus Limited .…”

Autonomous pollination of individual kiwifruit flowers: Toward a robotic kiwifruit pollinator

“…FCNs have been shown to be capable of operating in a range of dynamic environments and lighting conditions. Including previous work detecting kiwifruit in the real world for robotic harvesting (Williams et al, ). The network used was the Faster‐RCNN inception v2.0 model provided via Github .…”