An Adaptive $Q$-Learning Algorithm Developed for Agent-Based Computational Modeling of Electricity Market

Rahimiyan, Morteza; Mashhadi, Habib Rajabi

doi:10.1109/tsmcc.2010.2044174

Cited by 74 publications

(31 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In [15] and [16], RL approaches are proposed for learning bidding strategies in forward electricity markets. Reddy and Veloso [17] proposed a RL approach to learn pricing strategies for a broker agent in a tariff market.…”

Section: B Reinforcement Learning For Drmentioning

confidence: 99%

Reinforcement Learning of Heuristic EV Fleet Charging in a Day-Ahead Electricity Market

Vandael

Claessens

Ernst

et al. 2015

IEEE Trans. Smart Grid

169

View full text Add to dashboard Cite

Abstract-This paper addresses the problem of defining a day-ahead consumption plan for charging a fleet of electric vehicles (EVs), and following this plan during operation. A challenge herein is the beforehand unknown charging flexibility of EVs, which depends on numerous details about each EV (e.g., plug-in times, power limitations, battery size, power curve, etc.). To cope with this challenge, EV charging is controlled during opertion by a heuristic scheme, and the resulting charging behavior of the EV fleet is learned by using batch mode reinforcement learning. Based on this learned behavior, a cost-effective day-ahead consumption plan can be defined. In simulation experiments, our approach is benchmarked against a multistage stochastic programming solution, which uses an exact model of each EVs charging flexibility. Results show that our approach is able to find a day-ahead consumption plan with comparable quality to the benchmark solution, without requiring an exact day-ahead model of each EVs charging flexibility.Index Terms-Demand-side management, electric vehicles (EVs), reinforcement learning (RL), stochastic programming (SP). NOMENCLATUREThe symbols and notations used throughout this paper are summarized below. Manuscript received October 15, 2013; revised March 20, 2014 and August 28, 2014; accepted October 20, 2014 Offset from day-ahead prices, to define imbalance prices. Functions T(h)Mapping from market period h to the set of control periods in market period h. f heur Heuristic function to dispatch power to EVs. Real variables E da hEnergy bought in the day-ahead market for market period h. Energy charged by the EV fleet in control period t. i P n t Charging power of EV i during control period t in senario n.

show abstract

Section: B Reinforcement Learning For Drmentioning

confidence: 99%

Reinforcement Learning of Heuristic EV Fleet Charging in a Day-Ahead Electricity Market

Vandael

Claessens

Ernst

et al. 2015

IEEE Trans. Smart Grid

169

View full text Add to dashboard Cite

show abstract

“…To study the di erent aspects of the proposed method and its accuracy, three test networks utilized in [15,23,28] are used. The analysis will be done in three parts: at rst, using the network in [23], the accuracy of the proposed method in choosing the bid for the energy market will be investigated, and it will be shown that the proposed method is accurate similar to metaheuristic approaches, yet it is faster in solving bidding strategy problems in pool-based power markets.…”

Section: Simulation Resultsmentioning

confidence: 99%

“…The analysis will be done in three parts: at rst, using the network in [23], the accuracy of the proposed method in choosing the bid for the energy market will be investigated, and it will be shown that the proposed method is accurate similar to metaheuristic approaches, yet it is faster in solving bidding strategy problems in pool-based power markets. Then, using the network [28], application of the proposed method for multi-generation unit case is considered, and the capability of the proposed method is proven. Finally, using the network like [15], application of the proposed method for choosing the bid in the integrated energy and SR market will be investigated.…”

Section: Simulation Resultsmentioning

confidence: 99%

“…In addition, Eqs. (24) and (27) show the quantities of power and SR allocated to constrained generator i, respectively; (25) and (28) show the quantities of power and reserve allocated to non-constrained generator k, respectively. In these equations, coe cient i is the marginal cost of the capacity constraint.…”

Section: The Iso Problemmentioning

confidence: 99%

“…Table 5 shows generators' cost function parameters, which are like the network studied in [28]. Total demand is assumed to be 1000 MW.…”

Section: Energy Market-multi-generator Casementioning

confidence: 99%

See 2 more Smart Citations

A New Solution Approach for Supply Function Equilibrium-based Bidding Strategy in Electricity Markets

Hematabadi

Foroud

2017

Scientia Iranica

View full text Add to dashboard Cite

Abstract. This paper presents a new analytical solution method for Supply FunctionEquilibrium-based (SFE) bidding strategy in electricity markets. It is assumed that every Generation Company (GenCo) has some generation units and bids a Linear Supply Function (LSF) for each of its units to the Independent System Operator (ISO). The problem is modeled as a bi-level optimization problem; in the inner level, ISO clears the market to maximize social welfare; in the outer level, each GenCo tries to maximize its individual welfare. The proposed method is used to solve the outer-level problem using an iterative algorithm, in which LSF coe cients are parameterized. The formulation is developed for both elastic and non-elastic demands for single-and multi-generator cases and also for integrated energy and Spinning Reserve (SR) markets with three models of competitions: a-, b-, and k-parameterization. Three sample networks are used to evaluate the proposed method. The results show that the proposed method is e ective and accurate for GenCos' strategic bidding in electricity markets compared with other optimization algorithms.

show abstract

Adaptive control and signal processing literature survey (No. 21)

2010

Adaptive Control & Signal

View full text Add to dashboard Cite

In order to keep readers up-to-date, the journal will contain a list of recently published journal articles relevant to the fields of adaptive control and signal processing six times a year. This list is compiled from a wide range of journals. Please note that even though the list has been broadly categorized these classifications are by no means strict, and are there to assist the reader. Please also note that inclusion in the list is not an endorsement of a paper's quality. If you have any suggestions or comments, please e-mail Erfu Yang at erfu.yang@eee.strath.ac.uk. 1. ADAPTIVE SYSTEMS Adetola V, Guay M. Performance improvement in adaptive control of linearly parameterized nonlinear systems. IEEE Transactions on Automatic Control 2010; 55(9):2182-2186. Bartoszewicz A, Nowacka-Leverton A. ITAE optimal sliding modes for third-order systems with input signal and state constraints. IEEE Transactions on Automatic Control 2010; 55(8):1928-1932. Bresch-Pietri D, Krstic M. Delay-adaptive predictor feedback for systems with unknown long actuator delay. IEEE Transactions on Automatic Control 2010; 55(9):2106-2112. Chen W, Jiao L, Li R, Li J. Adaptive backstepping fuzzy control for nonlinearly parameterized systems with periodic disturbances. IEEE Transactions on Fuzzy Systems 2010; 18(4):674-685. Cheng C-C, Chien S-H, Shih F-C. Design of robust adaptive variable structure tracking controllers with application to rigid robot manipulators. Immersion and invariance adaptive control of nonlinearly parameterized nonlinear systems. IEEE Transactions on Automatic Control 2010; 55(9):2209-2214. Miller D, Mansouri N. Model reference adaptive control using simultaneous probing, estimation, and control. IEEE Transactions on Automatic Control 2010; 55(9):2014-2029. Mizumoto I, Ohdaira S, Iwai Z. Output feedback strict passivity of discrete-time nonlinear systems and adaptive control system design with a PFC. Automatica 2010; 46(9):1503-1509. Moreno-Valenzuela J, Santibáñez V, Orozco-Manrııquez E, González-Hernández L. Theory and experiments of global adaptive output feedback tracking control of manipulators. IET Control Theory and Applications 2010; 4(9):1639-1654. Plestan F, Shtessel Y, Brégeault V, Poznyak A. New methodologies for adaptive sliding mode control. International Journal of Control 2010; 83(9):1907-1919.Reinhartz-Berger I, Soffer P, Sturm A. Extending the adaptability of reference models.

show abstract

An Adaptive $Q$-Learning Algorithm Developed for Agent-Based Computational Modeling of Electricity Market

Cited by 74 publications

References 25 publications

Reinforcement Learning of Heuristic EV Fleet Charging in a Day-Ahead Electricity Market

Reinforcement Learning of Heuristic EV Fleet Charging in a Day-Ahead Electricity Market

A New Solution Approach for Supply Function Equilibrium-based Bidding Strategy in Electricity Markets

Adaptive control and signal processing literature survey (No. 21)

Contact Info

Product

Resources

About