Automation of bucket-filling is of crucial significance to the fully automated systems for wheel loaders. Most previous works are based on a physical model, which cannot adapt to the changeable and complicated working environment. Thus, in this paper, a data-driven reinforcement-learning (RL)-based approach is proposed to achieve automatic bucket-filling. An automatic bucket-filling algorithm based on Q-learning is developed to enhance the adaptability of the autonomous scooping system. A nonlinear, non-parametric statistical model is also built to approximate the real working environment using the actual data obtained from tests. The statistical model is used for predicting the state of wheel loaders in the bucket-filling process. Then, the proposed algorithm is trained on the prediction model. Finally, the results of the training confirm that the proposed algorithm has good performance in adaptability, convergence, and fuel consumption in the absence of a physical model. The results also demonstrate the transfer learning capability of the proposed approach. The proposed method can be applied to different machine-pile environments.

Accurate prediction of the throttle value and state for wheel loaders can help to achieve autonomous operation, thereby reducing the cost and accident rate. However, existing methods based on a physical model cannot accurately reflect the operator’s driving habits and the interaction between wheel loaders and the environment. In this paper, a deep-learning-based prediction model is developed to predict the throttle value and state for wheel loaders by learning from driving data. Multiple long–short-term memory (LSTM) networks are used to extract the temporal features of different stages during the operation of the wheel loader. Two backward-propagation neural networks (BPNNs), which use the temporal feature extracted by LSTM as the input, are designed to output the final prediction results of throttle value and state, respectively. The proposed prediction model is trained and tested using the data from two different conditions. The end-to-end LSTM prediction model and BPNNs are used as benchmark models. The results indicate that the proposed prediction model has good prediction accuracy and adaptability. Furthermore, the relationship between the prediction performance and signal sampling frequency is also studied. The proposed prediction method that combines driving data and deep learning can make the throttle action conform to the decisions of an experienced operator, providing technical support for the autonomous operation of construction machinery.