A Situationally Aware Voice‐commandable Robotic Forklift Working Alongside People in Unstructured Outdoor Environments

Abstract

One long-standing challenge in robotics is the realization of mobile autonomous robots able to operate safely in human workplaces, and be accepted by the human occupants. We describe the development of a multi-ton robotic forklift intended to operate alongside people and vehicles, handling palletized materials within existing, active outdoor storage facilities.The system has four novel characteristics. The first is a multimodal interface that allows users to efficiently convey task-level commands to the robot using a combination of pen-based gestures and natural language speech. These tasks include the manipulation, transport, and placement of palletized cargo within dynamic, human-occupied warehouses. The second is the robot's ability to learn the visual identity of an object from a single user-provided example and use the learned model to reliably and persistently detect objects despite significant spatial and temporal excursions. The third is a reliance on local sensing that allows the robot to handle variable palletized cargo and navigate within dynamic, minimally-prepared environments without GPS. The fourth concerns the robot's operation in close proximity to people, including its human supervisor, pedestrians who may cross or block its path, moving vehicles, and forklift operators who may climb inside the robot and operate it manually. This is made possible by interaction mechanisms that facilitate safe, effective operation around people.This paper provides a comprehensive description of the system's architecture and implementation, indicating how real-world operational requirements motivated key design choices. We offer qualitative and quantitative analyses of the robot operating in real settings and discuss the lessons learned from our effort.Put the pallet of pipes on the truck Figure 1: (left) The mobile manipulation platform is designed to safely coexist with people within unstructured environments while performing material handling under the direction of humans.

Inferring Compact Representations for Efficient Natural Language Understanding of Robot Instructions

“…These models are typically learned from annotated natural language corpora as well as through interaction with humans [29,36,37]. Probabilistic grounding models have been shown to be effective at interpreting cooking instructions [38], learning spatial relations in semantic maps [5,15], and directing mobile manipulators [39], among others.…”

Robust shared autonomy for mobile manipulation with continuous scene monitoring

“…Stückler et al [14] use motion key frames generated by an operator based on segmented perception data with the autonomous system interpolating between key frames given constraints. Walter et al [15] describe an autonomous forklift able to operate safely in a shared workspace with humans accepting task-level commands through natural language and pen-based gestures. A high level control method implementing shared autonomy for debris clearance is presented by Kaiser et al [16] where the system proposes affordances and actions for the operator to select and command.…”

Optimal sampling-based Feedback Motion Trees among obstacles for controllable linear systems with linear constraints

“…if J + Cost(z f ) < Jupper and ObstacleFree(z0, z f , e) then 5 Jupper ← J + Cost(z f ); z from ← z0; e best ← e; 6 return (z from , e best ); Algorithm 1 differs from the typical loop of local steering and collision checking in Line 3 and 5. Note that a TPBVP solution from z 0 to z f is discarded later if the trajectory to goals from z 0 via z f is not better than existing or best found trajectories from z 0 to goals.…”

“…In recent decades, an unprecedented level of autonomy has been progressively claimed by such randomized algorithms as Probabilistic RoadMaps (PRMs) [2] and RRTs [3]. However, robotic motion still needs to further utilize physical limits of the system, especially in expected future situations where robots are placed in challenges or need seamless cooperation with skilled co-workers [4], [5].…”