QuaRL: Quantization for Sustainable Reinforcement Learning

Lam, Maximilian; Chitlangia, Sharad; Wan, Zishen; Barth-Maron, Gabriel; Faust, Aleksandra; Reddi, Vijay Janapa

doi:10.48550/arxiv.1910.01055

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Post-training quantization (PTQ) takes a trained full-precision model (32-bit floating point, fp32) and quantizes (to an 8-bit representation, int8) its weights to lower-precision values [ 19 ]. Uniform affine quantization is used to transform the fp32 values into int8 ones [ 29 ]. The process consists of first extracting the maximum and minimum values in the fp32 working range.…”

Section: Methodsmentioning

confidence: 99%

Computational Optimization of Image-Based Reinforcement Learning for Robotics

Ferraro

Maele

Mazzaglia

et al. 2022

Sensors

View full text Add to dashboard Cite

The robotics field has been deeply influenced by the advent of deep learning. In recent years, this trend has been characterized by the adoption of large, pretrained models for robotic use cases, which are not compatible with the computational hardware available in robotic systems. Moreover, such large, computationally intensive models impede the low-latency execution which is required for many closed-loop control systems. In this work, we propose different strategies for improving the computational efficiency of the deep-learning models adopted in reinforcement-learning (RL) scenarios. As a use-case project, we consider an image-based RL method on the synergy between push-and-grasp actions. As a first optimization step, we reduce the model architecture in complexity, by decreasing the number of layers and by altering the architecture structure. Second, we consider downscaling the input resolution to reduce the computational load. Finally, we perform weight quantization, where we compare post-training quantization and quantized-aware training. We benchmark the improvements introduced in each optimization by running a standard testing routine. We show that the optimization strategies introduced can improve the computational efficiency by around 300 times, while also slightly improving the functional performance of the system. In addition, we demonstrate closed-loop control behaviour on a real-world robot, while processing everything on a Jetson Xavier NX edge device.

show abstract

Section: Methodsmentioning

confidence: 99%