Dyna-Q is a reinforcement learning method widely used in AGV path planning. However, in large complex dynamic environments, due to the sparse reward function of Dyna-Q and the large searching space, this method has the problems of low search efficiency, slow convergence speed, and even inability to converge, which seriously reduces the performance and practicability of it. To solve these problems, this paper proposes an Improved Dyna-Q algorithm for AGV path planning in large complex dynamic environments. First, to solve the problem of the large search space, this paper proposes a global path guidance mechanism based on heuristic graph, which can effectively reduce the path search space and, thus, improve the efficiency of obtaining the optimal path. Second, to solve the problem of the sparse reward function in Dyna-Q, this paper proposes a novel dynamic reward function and an action selection method based on the heuristic graph, which can provide more intensive feedback and more efficient action decision for AGV path planning, effectively improving the convergence of the algorithm. We evaluated our approach in scenarios with static obstacles and dynamic obstacles. The experimental results show that the proposed algorithm can obtain better paths more efficiently than other reinforcement-learning-based methods including the classical Q-Learning and the Dyna-Q algorithms.
With the development of industrial manufacturing intelligence, the role of rotating machinery in industrial production and life is more and more important. Aiming at the problems of the complex and changeable working environment of rolling bearings and limited computing ability, fault feature information cannot be effectively extracted, and the current deep learning model is difficult to be compatible with lightweight and high efficiency. Therefore, this paper proposes a fault detection method for power equipment based on an energy spectrum diagram and deep learning. Firstly, a novel two-dimensional time-frequency feature representation method and energy spectrum feature map based on wavelet packet transform is proposed, and an energy spectrum feature map dataset is made for subsequent diagnosis. This method can realize multi-resolution analysis, fully extract the feature information contained in the fault signal, and accelerate the convergence of the subsequent diagnosis model. Secondly, a lightweight residual dense convolutional neural network model (LR-DenseNet) is proposed. This model combines the advantages of residual learning and a dense connection, and can not only extract deep features more easily, but can also effectively use shallow features. Then, based on the lightweight residual dense convolutional neural network model, an LR-DenseSENet model is proposed. By introducing the transfer learning strategy and adding the channel domain, an attention mechanism is added to the channel feature fusion layer, with the accuracy of detection up to 99.4%, and the amount of parameter calculation greatly reduced to one-fifth of that of VGG. Finally, through an experimental analysis, it is verified that the fault detection model designed in this paper based on the combination of an energy spectrum feature map and LR-DenseSENet achieves a satisfactory detection effect.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.