Decoupled Visual Servoing With Fuzzy <i>Q</i>-Learning

Shi, Haobin; Li, Xuesi; Hwang, Kao-Shing; Pan, Wei; Xu, Genjiu

doi:10.1109/tii.2016.2617464

Cited by 101 publications

(47 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As visual information has proven its potential in image-based control for autonomous mobile robots [23] and unmanned aerial vehicle [24], [25], the obtained images are employed to represent the state in this paper, which is employed to predict action. The last state s t−1 is served as the input and consists of two parts, as shown in Fig.…”

Section: A State Representationmentioning

confidence: 99%

Active Object Detection With Multistep Action Prediction Using Deep Q-Network

Han

Liu²,

Sun³

et al. 2019

IEEE Trans. Ind. Inf.

View full text Add to dashboard Cite

In recent years, great success has been achieved in visual object detection, which is one of the fundamental tasks in the field of industrial intelligence. Most of existing methods have been proposed to deal with single well-captured still images, while in practical robotic applications, due to nuisances, such as tiny scale, partial view, or occlusion, one still image may not contain enough information for object detection. However, an intelligent robot has the capability to adjust its viewpoint to get better images for detection. Therefore, active object detection becomes a very important perception strategy for intelligent robots. In this paper, by formulating active object detection as a sequential action decision process, a deep reinforcement learning framework is established to resolve it. Furthermore, a novel deep Q-learning network (DQN) with a dueling architecture is proposed, the network has two separate output channels, one predicts action type and the other predicts action range. By combining the two output channels, the action space is explored more efficiently. Several methods are extensively validated and the results show that the proposed one obtains the best results and predicts action in real time.

show abstract

Section: A State Representationmentioning

confidence: 99%

Active Object Detection With Multistep Action Prediction Using Deep Q-Network

Han

Liu²,

Sun³

et al. 2019

IEEE Trans. Ind. Inf.

View full text Add to dashboard Cite

show abstract

“…, T − 1 is the real-time reward. Equations (8)- (10) gives the updating method of the state-action function.…”

Section: An Improved Q-learning Methods In Semi-markov Decision Processesmentioning

confidence: 99%

“…Reinforcement learning (RL) [9] is an effective machine learning method and the goal of reinforcement learning for an agent is to learn an optimal action strategy and obtain optimal rewards. This year, it has attracted extensive attention from scholars [10][11][12].…”

Section: The Robot Confrontation Systemmentioning

confidence: 99%

Fuzzy Reinforcement Learning and Curriculum Transfer Learning for Micromanagement in Multi-Robot Confrontation

Meng

2019

Information

View full text Add to dashboard Cite

Multi-Robot Confrontation on physics-based simulators is a complex and time-consuming task, but simulators are required to evaluate the performance of the advanced algorithms. Recently, a few advanced algorithms have been able to produce considerably complex levels in the context of the robot confrontation system when the agents are facing multiple opponents. Meanwhile, the current confrontation decision-making system suffers from difficulties in optimization and generalization. In this paper, a fuzzy reinforcement learning (RL) and the curriculum transfer learning are applied to the micromanagement for robot confrontation system. Firstly, an improved Qlearning in the semi-Markov decision-making process is designed to train the agent and an efficient RL model is defined to avoid the curse of dimensionality. Secondly, a multi-agent RL algorithm with parameter sharing is proposed to train the agents. We use a neural network with adaptive momentum acceleration as a function approximator to estimate the state-action function. Then, a method of fuzzy logic is used to regulate the learning rate of RL. Thirdly, a curriculum transfer learning method is used to extend the RL model to more difficult scenarios, which ensures the generalization of the decision-making system. The experimental results show that the proposed method is effective.

show abstract

“…The intelligent and autonomous agent means that each of these agents operates independently, and by using the reward/penalty that they receive from the environment can serve the main purpose of the game, which is reaching the game's answer. The Q-learning strategy, which is considered, to solve the RL-based game theory problem in this paper, is a model-free and a simple solution mechanism [39]. According to Figure 1, due to the nature of RL that learns the optimal control policy (OCP) by interacting with the environment [40], this paper first performs the offline simulations in which the intelligent agents use trial and error method to extract the OCP.…”

Section: Introductionmentioning

confidence: 99%

Assessing the Use of Reinforcement Learning for Integrated Voltage/Frequency Control in AC Microgrids

2020

View full text Add to dashboard Cite

The main purpose of this paper is to present a novel algorithmic reinforcement learning (RL) method for damping the voltage and frequency oscillations in a micro-grid (MG) with penetration of wind turbine generators (WTG). First, the continuous-time environment of the system is discretized to a definite number of states to form the Markov decision process (MDP). To solve the modeled discrete RL-based problem, Q-learning method, which is a model-free and simple iterative solution mechanism is used. Therefore, the presented control strategy is adaptive and it is suitable for the realistic power systems with high nonlinearities. The proposed adaptive RL controller has a supervisory nature that can improve the performance of any kind of controllers by adding an offset signal to the output control signal of them. Here, a part of Denmark distribution system is considered and the dynamic performance of the suggested control mechanism is evaluated and compared with fuzzy-proportional integral derivative (PID) and classical PID controllers. Simulations are carried out in two realistic and challenging scenarios considering system parameters changing. Results indicate that the proposed control strategy has an excellent dynamic response compared to fuzzy-PID and traditional PID controllers for damping the voltage and frequency oscillations.

show abstract

Decoupled Visual Servoing With Fuzzy Q-Learning

Cited by 101 publications

References 31 publications

Active Object Detection With Multistep Action Prediction Using Deep Q-Network

Active Object Detection With Multistep Action Prediction Using Deep Q-Network

Fuzzy Reinforcement Learning and Curriculum Transfer Learning for Micromanagement in Multi-Robot Confrontation

Assessing the Use of Reinforcement Learning for Integrated Voltage/Frequency Control in AC Microgrids

Contact Info

Product

Resources

About