A Fog Robotics Approach to Deep Robot Learning: Application to Object Recognition and Grasp Planning in Surface Decluttering

Abstract

The growing demand of industrial, automotive and service robots presents a challenge to the centralized Cloud Robotics model in terms of privacy, security, latency, bandwidth, and reliability. In this paper, we present a 'Fog Robotics' approach to deep robot learning that distributes compute, storage and networking resources between the Cloud and the Edge in a federated manner. Deep models are trained on non-private (public) synthetic images in the Cloud; the models are adapted to the private real images of the environment at the Edge within a trusted network and subsequently, deployed as a service for lowlatency and secure inference/prediction for other robots in the network. We apply this approach to surface decluttering, where a mobile robot picks and sorts objects from a cluttered floor by learning a deep object recognition and a grasp planning model. Experiments suggest that Fog Robotics can improve performance by sim-to-real domain adaptation in comparison to exclusively using Cloud or Edge resources, while reducing the inference cycle time by 4× to successfully declutter 86% of objects over 213 attempts.

Federated Learning for Internet of Things: A Comprehensive Survey

“…Through communication rounds, the server can accumulate a significant knowledge of different robots to build a powerful learning model, and thus the imitation learning efficiency and accuracy of local robots can be improved in comparison with centralized learning approaches. As an extended version of cloud computing, fog/edge computing is also helpful to provide low-latency communication for federated robotics [165]. Both computing, networking, and storage resources among robots are shared with a fog server for enabling federated learning with security awareness.…”

A Function as a Service Based Fog Robotic System for Cognitive Robots

“…It separated the tasks with a cloud for high level perception, an edge for time critical tasks, and a cloud-edge hybrid for the majority of robot applications. Tanwani et al [3] introduced a fog robotics approach for secure and distributed deep robot learning. It provides that deep learning models are trained on public synthetic images in the cloud, and the private real images are adapted at the edge within a trusted network which reduces the round-trip communication time for inference of a mobile robot.…”

“…However, such cloud robot services could give rise to security issues of privacy breaches and latency issues of control signal delays for robot motions. Recently, to solve these issues, fog robotics, distributing computing work properly with fog servers and edges, is receiving attention for its advantages in reducing latency and security matters (Figure 1) [3][4][5].…”

A Switching Offloading Mechanism for Path Planning and Localization in Robotic Applications

“…Open challenges in this area throughout the literature are concerned with developing adaptive multi-robot/machine control, capturing, modelling, predicting and anticipating the agent's interactions and designing distributed control and path planning algorithms that deliver flexible and safe working environments. Approaches similar to ours include [9], where gesture-based semaphore mirroring with a humanoid robot is split to remotely and locally executed functionality; [10], in which the authors identify a three-layered environment (Robot, Edge and Cloud) to overcome the challenges of network limits in a Deep Robot Learning application and [11] where Dew Robotics is introduced; this concept posits that critical computations are executed locally so that the robot can always react properly, while less critical tasks are moved to the Fog and Cloud, so to exploit the larger availability in computing, storage, and power supply. However, none of the aforementioned offloading decision schemes addresses the dynamic nature of the robot's environment.…”