This paper presents a method of Q-learning to solve the discounted linear quadratic regulator (LQR) problem for continuous-time (CT) continuous-state systems. Most available methods in the existing literature for CT systems to solve the LQR problem generally need partial or complete knowledge of the system dynamics. Q-learning is effective for unknown dynamical systems, but has generally been well understood only for discrete-time systems. The contribution of this paper is to present a Q-learning methodology for CT systems which solves the LQR problem without having any knowledge of the system dynamics. A natural and rigorous justified parameterization of the Q-function is given in terms of the state, the control input, and its derivatives. This parameterization allows the implementation of an online Q-learning algorithm for CT systems. The simulation results supporting the theoretical development are also presented.
After nearly a century with internal combustion engines dominating the transportation sector, it now appears that electric vehicles (EVs) are on the brink of enjoying rapid development due to numerous useful features they possess, such as less operational cost and reduced carbon emissions. EVs can act as load as well as source, by utilizing the technique known as Vehicle-to-Grid (or Grid-to-Vehicle technique if EVs are used as a load). This technique adds key features to an industrial microgrid in the form of primary frequency control and congestion management. In this paper, two controllers (grid regulation and charger controller) are proposed by considering different charging profiles, state of charge of electric vehicle batteries, and a varying number of electric vehicles in an electric vehicle fleet. These controllers provide bidirectional power flow, which can provide primary frequency control during different contingencies that an industrial microgrid may face during a 24-hour period. Simulation results prove that the proposed controllers provide reliable support in terms of frequency regulation to an industrial microgrid during contingencies. Furthermore, simulation results also depict that by adding more electric vehicles in the fleet during the vehicle-to-grid mode, the frequency of an industrial microgrid can be improved to even better levels. Different case studies in this article constitute an industrial microgrid with varied distributed energy resources (i.e. solar and wind farm), electric vehicles fleet, industrial and residential load along with diesel generator. These test cases are simulated and results are analyzed by using MATLAB/SIMULINK.
Energy storage system (ESS) possesses tremendous potential to counter both the rapid growth of intermittent renewable energy resources (RESs) and provide frequency support to the microgrid (MG). Since the deployment of ESS has overcome the imbalance between generation and consumption, however, their massive cost, as well as degradation tendency, are the restricting considerations that demand alternative solutions to provide stable microgrid operation. To assist ESS, the electric vehicles (EVs) are incorporated into the system. EVs have been gradually commercially viable and considerable focus has been paid to vehicle-to-grid technologies. Appropriate collaboration between ESS and EVs has good capability to manage the frequency irregularities to ensure the efficient operation of the MG. This article presents a novel combination of two control techniques i.e., model predictive control (MPC) and adaptive droop control (ADC), to tackle the frequency regulation issue in the isolated MG, by effectively controlling the ESS and EVs during the large-scale integration of RESs or huge change in load demand. Firstly, the MPC regulates the ESS according to the system frequency deviation, and secondly, the ADC manages the power of EVs according to system specifications by retaining the least possible power for potential usage of EVs. Moreover, an advanced genetic algorithm is applied to tune the MPC and ADC parameters in order to achieve optimized performance. An isolated MG is modeled and verified in MATLAB/Simulink using the above-mentioned control techniques. Further, different case studies are taken into account to validate the combination of ADC and MPC for frequency regulation of an isolated MG. Additionally, the proposed MPC controller is compared with fuzzy logic proportional-integral (FPI) controller and proportional-integral (PI) controller, the MPC provides better performance results as compared with FPI and PI controllers. INDEX TERMS Electric vehicles, adaptive droop control, energy storage system, model predictive control, frequency regulation, GA optimization technique.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.