Reinforcement Learning Control of a Forestry Crane Manipulator

Andersson, Jennifer; Bodin, Kenneth; Lindmark, Daniel; Servin, Martin; Wallin, Erik

doi:10.1109/iros51168.2021.9636219

Cited by 16 publications

(9 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Machines and their actuators come in different sizes and characteristics, so it is difficult, if not virtually impossible, to spend hours or days experimenting with each individual machine to collect sufficient data for learning a dynamics model, and to do so within the safety limits. This is contrary to studies in simulation [12], [13], where generating up to millions of interactions is cheap with regards to computation time, energy, and safety. Fortunately, recent studies have provided frameworks for learning controllers from surrogate Gaussian process (GP) models [14] trained on limited amount of data [8].…”

mentioning

confidence: 76%

“…The trend towards utilizing machine learning (ML) for complex control applications in the field of heavy-duty machines is becoming more and more visible in recent years [3], [4], [7], [12], [13], [21]. Some studies are carried out either entirely in simulation [12], [13], which can involve learning very complex tasks from up to millions of steps in simulation, or demonstrate control experiments on the real machine, where some form of model-based optimization [11], [15] or learning from demonstration [5], [6] is studied. ML is a less well-explored field of research compared to classical control, and safety considerations are a key limiting factor in testing new methodologies on heavy machines.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Nonlinear Model Learning for Compensation and Feedforward Control of Real-World Hydraulic Actuators Using Gaussian Processes

Taheri

Pelle²,

Rosth³

et al. 2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

This paper presents a robust machine learning framework for modeling and control of hydraulic actuators. We identify several important challenges concerning learning accurate models of the dynamics for real machines, including noise and uncertainty in state measurements, nonlinear effects, input delays, and dataefficiency. In particular, we propose a dual-Gaussian process (GP) model architecture to learn a surrogate dynamics model of the actuator, and showcase the accuracy of predictions against the piecewise and neural network models that have been widely used in the literature. In addition, we provide robust techniques for learning neural network inverse models and controllers by batch GP inference in an automated, seamless and computationally fast manner. Finally, we demonstrate the performance of the trained controllers in real-world feedforward and tracking control applications.Index Terms-Hydraulic actuators, machine learning for robot control, model learning for control. I. INTRODUCTIONA LL real-world mechanical systems that provide a means to automation today are powered by integrated controllers. These controllers are carefully designed and fine-tuned to provide the best possible performance for the intended application, and their development follows a conventional process. First, the system dynamics are modelled, wherein prior knowledge and general understanding of the physics of the underlying process Manuscript

show abstract

mentioning

confidence: 76%

Section: Related Workmentioning

confidence: 99%

Nonlinear Model Learning for Compensation and Feedforward Control of Real-World Hydraulic Actuators Using Gaussian Processes

Taheri

Pelle²,

Rosth³

et al. 2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…This is especially important in the field of heavy-duty machines, where dynamics are complex, interaction is slow and time-consuming, and tolerance is tight for unsafe exploration. Recently, there has been a surge in applying learning methods to heavy-duty machines [6], [7], [22]- [25] to discover automated, energyefficient [26] solutions for complex hydraulic systems.…”

Section: Related Workmentioning

confidence: 99%

“…3) is quite slow for the machine. Moreover, the optimization benchmarks exclude the time it takes to learn the GP hyperparameters (6), around ∼20s performed once for all algorithms, and the one-time caching operation required for BAGEL's fast predictions [19] takes ∼0.6s.…”

Section: A Experiments 1: Single-goal Controllermentioning

confidence: 99%

GPU-Accelerated Policy Optimization via Batch Automatic Differentiation of Gaussian Processes for Real-World Control

Taheri¹,

Pajarinen²,

Ghabcheloo³

2022

Preprint

View full text Add to dashboard Cite

The ability of Gaussian processes (GPs) to predict the behavior of dynamical systems as a more sample-efficient alternative to parametric models seems promising for realworld robotics research. However, the computational complexity of GPs has made policy search a highly time and memory consuming process that has not been able to scale to larger problems. In this work, we develop a policy optimization method by leveraging fast predictive sampling methods to process batches of trajectories in every forward pass, and compute gradient updates over policy parameters by automatic differentiation of Monte Carlo evaluations, all on GPU. We demonstrate the effectiveness of our approach in training policies on a set of reference-tracking control experiments with a heavy-duty machine. Benchmark results show a significant speedup over exact methods and showcase the scalability of our method to larger policy networks, longer horizons, and up to thousands of trajectories with a sublinear drop in speed.

show abstract

“…[16], where it is used, based on cameras, lidar, and motion and force sensors, to perform bucket loading of fragmented rock with a multi-objective target, including maximisation of the bucket loading; Ref. [17], where it is trained for the motion control of a forestry crane while minimising energy consumption; and Ref. [18], where it is used for the trajectory tracking control of an excavator arm, with the controller generating the valve-control signals directly.…”

Section: Introductionmentioning

confidence: 99%

Multi-Chamber Actuator Mode Selection through Reinforcement Learning–Simulations and Experiments

et al. 2022

View full text Add to dashboard Cite

This paper presents the development and implementation of a reinforcement learning agent as the mode selector for a multi-chamber actuator in a load-sensing architecture. The agent selects the mode of the actuator to minimise system energy losses. The agent was trained in a simulated environment and afterwards deployed to the real system. Simulation results indicated the capability of the agent to reduce energy consumption, while maintaining the actuation performance. Experimental results showed the capability of the agent to learn via simulation and to control the real system.

show abstract

Reinforcement Learning Control of a Forestry Crane Manipulator

Cited by 16 publications

References 11 publications

Nonlinear Model Learning for Compensation and Feedforward Control of Real-World Hydraulic Actuators Using Gaussian Processes

Nonlinear Model Learning for Compensation and Feedforward Control of Real-World Hydraulic Actuators Using Gaussian Processes

GPU-Accelerated Policy Optimization via Batch Automatic Differentiation of Gaussian Processes for Real-World Control

Multi-Chamber Actuator Mode Selection through Reinforcement Learning–Simulations and Experiments

Contact Info

Product

Resources

About