Tiny Robot Learning: Challenges and Directions for Machine Learning in Resource-Constrained Robots

Neuman, Sabrina M.; Plancher, Brian; Duisterhof, Bardienus P.; Banbury, Colby; Mazumder, Mark; Prakash, Shvetank; Jabbour, Jason; Faust, Aleksandra; Croon, Guido C. H. E. de; Reddi, Vijay Janapa

doi:10.1109/aicas54282.2022.9870000

Cited by 19 publications

(9 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In future work, we hope to deploy our approach onto physical robot hardware and test our approach in the context of real-world edge RL. Finally, we hope to apply our approach in the context of tiny robot learning [43] and help usher in a new era of ubiquitous edge RL.…”

Section: Discussionmentioning

confidence: 99%

Just Round: Quantized Observation Spaces Enable Memory Efficient Learning of Dynamic Locomotion

Grossman¹,

Plancher²

2022

Preprint

View full text Add to dashboard Cite

Deep reinforcement learning (DRL) is one of the most powerful tools for synthesizing complex robotic behaviors. But training DRL models is incredibly compute and memory intensive, requiring large training datasets and replay buffers to achieve performant results. This poses a challenge for the next generation of field robots that will need to learn on the edge to adapt to their environment. In this paper, we begin to address this issue through observation space quantization. We evaluate our approach using four simulated robot locomotion tasks and two state-of-the-art DRL algorithms, the on-policy Proximal Policy Optimization (PPO) and off-policy Soft Actor-Critic (SAC) and find that observation space quantization reduces overall memory costs by as much as 4.2× without impacting learning performance.

show abstract

Section: Discussionmentioning

confidence: 99%

Just Round: Quantized Observation Spaces Enable Memory Efficient Learning of Dynamic Locomotion

Grossman¹,

Plancher²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…This limits the deployment of DRL solutions on resource-constrained devices like autonomous robots. Tiny Machine Learning (TinyML) is a promising strategy to reduce the computational resources required for the deployment of robot learning approaches [4].…”

Section: Introductionmentioning

confidence: 99%

Neutrons Sensitivity of Deep Reinforcement Learning Policies on EdgeAI Accelerators

Bodmann,

Saveriano,

Kritikakou

et al. 2024

IEEE Trans. Nucl. Sci.

View full text Add to dashboard Cite

Autonomous robots and their application are becoming popular in several different fields, including tasks where robots closely interact with humans. Therefore, the reliability of computation must be paramount. In this work, we measure the reliability of Google's Coral Edge TPU executing three Deep Reinforcement Learning (DRL) models through an accelerated neutrons beam. We experimentally collect data that, when scaled to the natural neutron flux, accounts for more than 5 million years. Based on our extensive evaluation, we quantify and qualify the radiation-induced corruption on the correctness of DRL. Crucially, our data shows that the Edge TPU executing DRL has an error rate that is up to 18 times higher the limit imposed by international reliability standards. We found that, despite the feedback and intrinsic redundancy of DRL, the propagation of the fault induces the model to fail in the vast majority of cases or the model manages to finish but reports wrong metrics (i.e. speed, final position, reward). We provide insights on how radiation corrupts the model, on how the fault propagates in the computation, and about the failure characteristic of the controlled robot.

show abstract

“…Since the FCS is the basic platform for UAV, an in-depth study on isolated FMS rather than FCS could lead to lower research expenses, although prior experiments have failed to achieve this outcome. Nevertheless, the design of high-reliable heterogeneous FMS depends on many constraints, such as the requirements of being real-time, low-cost, low-power, high-performance, and minimal risk [11]. Another critical factor in the designing phase is the consideration of SWaP-C, which is an acronym for Size, Weight, Power, and Cost.…”

Section: Introductionmentioning

confidence: 99%

Heterogeneous Flight Management System (FMS) Design for Unmanned Aerial Vehicles (UAVs): Current Stages, Challenges, and Opportunities

Wang

et al. 2023

Drones

View full text Add to dashboard Cite

In the Machine Learning (ML) era, faced with challenges, including exponential multi-sensor data, an increasing number of actuators, and data-intensive algorithms, the development of Unmanned Aerial Vehicles (UAVs) is standing on a new footing. In particular, the Flight Management System (FMS) plays an essential role in UAV design. However, the trade-offs between performance and SWaP-C (Size, Weight, Power, and Cost) and reliability–efficiency are challenging to determine for such a complex system. To address these issues, the identification of a successful approach to managing heterogeneity emerges as the critical question to be answered. This paper investigates Heterogeneous Computing (HC) integration in FMS in the UAV domain from academia to industry. The overview of cross-layer FMS design is firstly described from top–down in the abstraction layer to left–right in the figurative layer. In addition, the HC advantages from Light-ML, accelerated Federated Learning (FL), and hardware accelerators are highlighted. Accordingly, three distinct research focuses detailed with visual-guided landing, intelligent Fault Diagnosis and Detection (FDD), and controller-embeddable Power Electronics (PE) to distinctly illustrate advancements of the next-generation FMS design from sensing, and computing, to driving. Finally, recommendations for future research and opportunities are discussed. In summary, this article draws a road map that considers the heterogeneous advantages to conducting the Flight-Management-as-a-Service (FMaaS) platform for UAVs.

show abstract

Tiny Robot Learning: Challenges and Directions for Machine Learning in Resource-Constrained Robots

Cited by 19 publications

References 32 publications

Just Round: Quantized Observation Spaces Enable Memory Efficient Learning of Dynamic Locomotion

Just Round: Quantized Observation Spaces Enable Memory Efficient Learning of Dynamic Locomotion

Neutrons Sensitivity of Deep Reinforcement Learning Policies on EdgeAI Accelerators

Heterogeneous Flight Management System (FMS) Design for Unmanned Aerial Vehicles (UAVs): Current Stages, Challenges, and Opportunities

Contact Info

Product

Resources

About