This is the accepted version of the paper.This version of the publication may differ from the final published version. Permanent repository link City Research OnlineAbstract --Three-phase grid-connected converters are widely used in renewable and electric power system applications. Traditionally, grid-connected converters are controlled with standard decoupled d-q vector control mechanisms. However, recent studies indicate that such mechanisms show limitations in their applicability to dynamic systems. This paper investigates how to mitigate such restrictions using a neural network to control a grid-connected rectifier/inverter. The neural network implements a dynamic programming algorithm and is trained by using backpropagation through time. To enhance performance and stability under disturbance, additional strategies are adopted, including the use of integrals of error signals to the network inputs and the introduction of grid disturbance voltage to the outputs of a well-trained network. The performance of the neural network controller is studied under typical vector control conditions and compared against conventional vector control methods, which demonstrates that the neural vector control strategy proposed in this paper is effective. Even in dynamic and power converter switching environments, the neural vector controller shows strong ability to trace rapidly changing reference commands, tolerate system disturbances, and satisfy control requirements for a faulted power system.

This is the accepted version of the paper.This version of the publication may differ from the final published version. The paper presents an analytical study on the optimal control of GCCs, including theoretically ideal optimal and suboptimal controllers. To overcome the inapplicability of the optimal GCC controller under practical conditions, a new RNN controller with an improved input structure is proposed to approximate the ideal optimal controller. The performance of an ideal optimal controller and a well-trained RNN controller was compared in close to real-life power converter switching environments, demonstrating that the proposed RNN controller can achieve close to ideal optimal control performance, even under low sampling rate conditions. The excellent performance of the proposed RNN controller under challenging and distorted system conditions further indicates the feasibility of using a RNN to approximate optimal control in practical applications. Permanent repository linkIndex Terms -optimal control, recurrent neural network, Levenberg-Marquardt, Forward Accumulation Through Time, Jacobian matrix, Backpropagation Through Time, dynamic programming, d-q vector control, grid-connected converter I. INTRODUCTION N modern electric power systems, power electronic converters play an increasingly important role in the integration of smart grids, renewable energy resources and energy storage devices (Fig. 1) A critical issue for energy generation from renewable sources and for smart grid integration is the control of the GCC (green boxes in Fig. 1). Traditionally, this type of converter is controlled using a standard decoupled d-q vector control approach [4]. However, recent studies have noted the limitations of the standard vector controller [4]. Practically, these limitations could result in low power quality, inefficient power generation and transmission, and a possible loss of electricity, all of which cause loss of dollars for both electric utility companies and electric energy customers. Recent research [5] has shown that recurrent neural networks (RNNs) can be trained and used to control gridconnected converters. In [5], the RNN implemented a dynamic programming (DP) algorithm and was trained using Backpropagation Through Time (BPTT). BPTT was combined with Resilient Propagation (RPROP) to accelerate the training. Compared to conventional standard vector control methods, the neural network vector controller produced an extremely fast response time, low overshoot, and, in general, the best performance [6]. In [7], it was shown that the neural network vector control technique can be extended to other applications, such as brushless dc motor drives.For both applications, conventional control techniques, such as PID and predictive control, were integrated into the DP-based neural network design [5]-[7]. This unifying approach produced some important advantages, including zero steady-state error, great control under physical system constraints, and the ability to exhibit adaptive control

We describe an Adaptive Dynamic Programming algorithm VGL(λ) for learning a critic function over a large continuous state space. The algorithm, which requires a learned model of the environment, extends Dual Heuristic Dynamic Programming to include a bootstrapping parameter analogous to that used in the reinforcement learning algorithm TD(λ). We provide on-line and batch mode implementations of the algorithm, and summarise the theoretical relationships and motivations of using this method over its precursor algorithms Dual Heuristic Dynamic Programming and TD(λ). Experiments for control problems using a neural network and greedy policy are provided.

No abstract

No abstract

This is the accepted version of the paper.This version of the publication may differ from the final published version. Permanent repository link AbstractWe present a recurrent neural-network (RNN) controller designed to solve the tracking problem for control systems. We demonstrate that a major difficulty in training any RNN is the problem of exploding gradients, and we propose a solution to this in the case of tracking problems, by introducing a stabilization matrix and by using carefully constrained context units. This solution allows us to achieve consistently lower training errors, and hence allows us to more easily introduce adaptive capabilities. The resulting RNN is one that has been trained off-line to be rapidly adaptive to changing plant conditions and changing tracking targets.The case study we use is a renewable-energy generator application; that of producing an efficient controller for a three-phase grid-connected converter. The controller we produce can cope with random variation of system parameters and fluctuating grid voltages. It produces tracking control with almost instantaneous response to changing reference states, and virtually zero oscil-$ This work was supported in part by the U.S. National Science Foundation under Grant EECS 1059265/1102159, the Mary K. Finley Missouri Endowment, and the Missouri S&T Intelligent Systems Center.Email addresses: michael.fairbank@virgin.net (Michael Fairbank), sli@eng.ua.edu (Shuhui Li), xfu@crimson.ua.edu (Xingang Fu), E.Alonso@city.ac.uk (Eduardo Alonso), dwunsch@mst.edu (Donald Wunsch) Preprint submitted to Neural Networks September 23, 2013 lation. This compares very favorably to the classical proportional integrator (PI) controllers, which we show produce a much slower response and settling time. In addition, the RNN we propose exhibits better learning stability and convergence properties, and can exhibit faster adaptation, than is achievable with adaptive critic designs.

Abstract-This paper gives specific divergence examples of value-iteration for several major Reinforcement Learning and Adaptive Dynamic Programming algorithms, when using a function approximator for the value function. These divergence examples differ from previous divergence examples in the literature, in that they are applicable for a greedy policy, i.e. in a "value iteration" scenario. Perhaps surprisingly, with a greedy policy, it is also possible to get divergence for the algorithms TD (1) and Sarsa(1). In addition to these divergences, we also achieve divergence for the Adaptive Dynamic Programming algorithms HDP, DHP and GDHP.

This is the accepted version of the paper.This version of the publication may differ from the final published version. This paper investigates how to mitigate such problems using a neural network to control a grid-connected rectifier/inverter. The neural network implements a dynamic programming (DP) algorithm and is trained using backpropagation through time. Permanent repository linkThe performance of the DP-based neural controller is studied for typical vector control conditions and compared with conventional vector control methods. The paper also investigates how varying grid and power converter system parameters may affect the performance and stability of the neural control system. Future research issues regarding the control of grid-connected converters using DP-based neural networks are analyzed. Index Terms -grid-connected rectifier/inverter, decoupled vector control, renewable energy conversion systems, neural controller, dynamic programming, backpropagation through time I. INTRODUCTIONN renewable and electric power system applications, a three-phase grid-connected dc/ac voltage-source PWM converter is usually employed to interface between the dc and ac systems. Typical converter configurations containing the grid-connected converter (GCC) include: 1) a dc/dc/ac converter for solar, battery and fuel cell applications [1,2], 2) a dc/ac converter for STATCOM applications [3,4], and 3) an ac/dc/ac converter for wind power and HVDC applications [4][5][6][7][8]. Figure 1 demonstrates the grid-connected dc/ac converter used in a microgrid to connect distributed energy resources. Conventionally, this type of converters is controlled using the standard decoupled d-q vector control approach [5][6][7][8].Notwithstanding its merits, recent studies indicate that the conventional vector control strategy is inherently limited [9,10], particularly when facing uncertainties [11]. For instance,