Reinforcement Learning Based Supplier-Agents for Electricity Markets

Abstract

Bidding strategies play important roles in maximizing the profits of power suppliers in competitive electricity markets. Therefore, it will be an advantage for a supplier to search for optimal bidding strategies in the market. In this paper the problem of designing Fuzzy Reinforcement Learning (FRL) supplier-agents that compete in forward electricity markets (e.g. Day-Ahead energy market) to maximize their revenues is studied. An IEEE 30-bus power system with 6 generators (supplier-agents) and three demand areas with stochastic loads are used for our simulation studies. This model is applicable to different types of commodity markets with numerous supply and demand agents.Index Terms--Bidding strategy, electricity market, fuzzy reinforcement learning, supplier agents.

Testing optimality of firms' actual bid in Iran wholesale electricity market

“…This method can model the dynamics of systems that are not in equilibrium. Agent‐based models can be categorized in terms of different learning algorithms such as model‐based adaptation algorithms, genetic algorithms, q‐learning, computational learning, and colony optimization; Azadeh et al, Kumar et al, Mahvi et al, and Rahimi‐Kian et al model the optimal bidding strategy of firms with this approach.…”

Modeling the Supplier Agent's Risk Strategy based on Fuzzy Logic Combined with the Q-Learning Algorithm

“…Using the multi-agent based on reinforcement learning, the market behavior under the Automatic Mitigation Procedure and the congestion management rules was studied in [2,3] respectively. In reference [4], using the fuzzy reinforcement learning method the supplier agents find the optimal bidding strategy for maximizing their revenues in the forward electricity market. The agent's risk strategy is adjusted using assumed fix parameters.…”

The Strategic Games Matrix as a Framework for Intelligent Autonomous Agents Hierarchical Control Strategies Modeling

“…A wide diversity of multi-controller and coordination problems has been treated recently, e.g., multiple mobile agents moving coordination and control (Shi, Wang and Chu, 2005), traffic congestion control (Alpcan and Başar, 2002), multiple mobile robot control (Shao, Xie, Yu and Wang, 2005), collision avoidance scheme in navigation control (Dimaragonas and Kyriakopoulus, 2005), secure routing in communication networks (Bohacek, Hespanha and Obraczka, 2002), optimal bidding strategies in the electricity market (Rahimi-Kian, Tabarraei and Sadeghi, 2005), automa-teams coordination and control (Liu, Galati and Simaan, 2004), attack and deception strategies in military operations (Castañón, Pachter and Chandler, 2004), and intrusion detection in access control systems (Alpcan and Başar, 2004).…”