Adaptation of a wheel loader automatic bucket filling neural network using reinforcement learning

Dadhich, Siddharth; Sandin, Fredrik; Bodin, Ulf; Andersson, Ulf; Martinsson, Torbjörn

doi:10.1109/ijcnn48605.2020.9206849

Cited by 17 publications

(9 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Artificial intelligence (AI)-based methods include fuzzy logic for wheel-loader action selection [15] and digging control [16] by using feed-forward neural networks to model digging resistance and machine dynamics. Automatic bucket filling by learning from demonstration was recently demonstrated in [17][18][19] and extended in [20] with a reinforcement learning algorithm for automatic adaptation of an already-trained model to a new pile of different soil. The imitation model in [17] is a time-delayed neural network that predicts the lift and tilt actions of joysticks during the filling of a bucket; it was trained with 100 examples from an expert operator and used no information about the material or the pile.…”

Section: Related Work and Our Contributionmentioning

confidence: 99%

“…The imitation model in [17] is a time-delayed neural network that predicts the lift and tilt actions of joysticks during the filling of a bucket; it was trained with 100 examples from an expert operator and used no information about the material or the pile. With the adaptation algorithm in [20], the network adapts from loading mediumcoarse gravel to cobble gravel, with a five-to ten-percent increase in bucket filling after 40 loadings. The first use of reinforcement learning to control a scooping mechanism was recently published [21].…”

Section: Related Work and Our Contributionmentioning

confidence: 99%

See 1 more Smart Citation

Continuous Control of an Underground Loader Using Deep Reinforcement Learning

et al. 2021

View full text Add to dashboard Cite

The reinforcement learning control of an underground loader was investigated in a simulated environment by using a multi-agent deep neural network approach. At the start of each loading cycle, one agent selects the dig position from a depth camera image of a pile of fragmented rock. A second agent is responsible for continuous control of the vehicle, with the goal of filling the bucket at the selected loading point while avoiding collisions, getting stuck, or losing ground traction. This relies on motion and force sensors, as well as on a camera and lidar. Using a soft actor–critic algorithm, the agents learn policies for efficient bucket filling over many subsequent loading cycles, with a clear ability to adapt to the changing environment. The best results—on average, 75% of the max capacity—were obtained when including a penalty for energy usage in the reward.

show abstract

Section: Related Work and Our Contributionmentioning

confidence: 99%

Section: Related Work and Our Contributionmentioning

confidence: 99%

Continuous Control of an Underground Loader Using Deep Reinforcement Learning

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Because simulation models are not derived from the real world, RL-based simulation cannot learn features of the real world well. Dadhich et al [5] used RL to achieve the automatic bucket-filling of wheel loaders through real-time interaction with the real environment. However, interacting with the real environment to train the RL algorithm is costly and time-consuming.…”

Section: Related Workmentioning

confidence: 99%

“…Bucket-filling is a relatively repetitive task for the operators of wheel-loaders and is suitable for automation. Automatic bucket-filling is also required for efficient remote operation and the development of fully autonomous solutions [5]. The interaction condition between the bucket and the pile strongly affects the bucket-filling.…”

Section: Introductionmentioning

confidence: 99%

Data-Driven Reinforcement-Learning-Based Automatic Bucket-Filling for Wheel Loaders

et al. 2021

View full text Add to dashboard Cite

Automation of bucket-filling is of crucial significance to the fully automated systems for wheel loaders. Most previous works are based on a physical model, which cannot adapt to the changeable and complicated working environment. Thus, in this paper, a data-driven reinforcement-learning (RL)-based approach is proposed to achieve automatic bucket-filling. An automatic bucket-filling algorithm based on Q-learning is developed to enhance the adaptability of the autonomous scooping system. A nonlinear, non-parametric statistical model is also built to approximate the real working environment using the actual data obtained from tests. The statistical model is used for predicting the state of wheel loaders in the bucket-filling process. Then, the proposed algorithm is trained on the prediction model. Finally, the results of the training confirm that the proposed algorithm has good performance in adaptability, convergence, and fuel consumption in the absence of a physical model. The results also demonstrate the transfer learning capability of the proposed approach. The proposed method can be applied to different machine-pile environments.

show abstract

“…Furthermore, machine control and perception strategies use software algorithms, datasets, and data fusion which also place high computational hardware requirements [6]. Additionally, methods such as imitation learning (IL) are used to construct surrogate models for control of autonomous HDMM, wherein a human-operator demonstrates a task, which is subsequently "imitated" by the model [33]. The human performance may differ and thus, work performance evaluation models are valuable to demonstrate ideal working methods for IL.…”

Section: ) Sensorsmentioning

confidence: 99%

Autonomous Heavy-Duty Mobile Machinery: A Multidisciplinary Collaborative Challenge

Machado

Fassbender

Taheri

et al. 2021

2021 IEEE International Conference on Technology and Entrepreneurship (ICTE)

View full text Add to dashboard Cite

Heavy-duty mobile machines (HDMMs) are a wide range of machinery used in diverse and critical application areas which are currently facing several issues like skilled labor shortage, poor safety records, and harsh work environments. Consequently, efforts are underway to increase automation in HDMMs for increased productivity and safety, eventually transitioning to operator-less autonomous HDMMs to address skilled labor shortages. However, HDMM are complex machines requiring continuous physical and cognitive inputs from human-operators. Thus, developing autonomous HDMM is a huge challenge, with current research and developments being performed in several independent research domains. Through this study, we use the bounded rationality concept to propose multidisciplinary collaborations for new autonomous HDMMs and apply the transaction cost economics framework to suggest future implications in the HDMM industry. Furthermore, we introduce a conceptual understanding of collaborations in the autonomous HDMM as a unified approach, while highlighting the practical implications and challenges of the complex nature of such multidisciplinary collaborations. The collaborative challenges and potentials are mapped out between the following topics: mechanical systems, AI methods, software systems, sensors, connectivity, simulations and process optimization, business cases, organization theories, and finally, regulatory frameworks.

show abstract

Adaptation of a wheel loader automatic bucket filling neural network using reinforcement learning

Cited by 17 publications

References 22 publications

Continuous Control of an Underground Loader Using Deep Reinforcement Learning

Continuous Control of an Underground Loader Using Deep Reinforcement Learning

Data-Driven Reinforcement-Learning-Based Automatic Bucket-Filling for Wheel Loaders

Autonomous Heavy-Duty Mobile Machinery: A Multidisciplinary Collaborative Challenge

Contact Info

Product

Resources

About