Research on Bidding Strategy of Thermal Power Companies in Electricity Market Based on Multi-Agent Deep Deterministic Policy Gradient

Abstract

With the continuous improvement of new energy penetration in the power system, the price of the spot market of power frequently fluctuates greatly, which damages the income of a large number of thermal power enterprises. In order to lock in the profit, thermal power enterprises should turn the main target of profit to the medium and long-term power market. With the continuous advancement of the reform in China's power system, major changes have taken place in the medium and long-term power transactions, including the transaction target, organization method, clearing method and so on, so it is urgent to explore the quotation strategy of thermal power enterprises under the medium and long term market changes. Based on the theory of game equilibrium, this paper establishes non-cooperative game and cooperative game models between thermal power companies. Considering that the traditional reinforcement learning method is difficult to solve the multi-agent incomplete information game model, this paper uses the Multi-Agent Deep Deterministic Policy Gradient(MADDPG) algorithm to solve the above model. Finally, the validity of the proposed model is proved by a numerical example. The results show that, compared with other reinforcement learning algorithms, when solving the multi-agent incomplete information game model, the quotation obtained by MADDPG is more accurate, the revenue is increased by 5.2%, and the convergence time is reduced by 50%.In addition, this paper finds that in the medium and long-term power market, thermal power companies are more inclined to use physical retention methods to make profits. The greater the market power of thermal power companies, the greater the probability of physical retention. When low-cost thermal power companies retain more power, they will increase market clearing electricity prices and harm market efficiency. Regulators should focus on the market behavior of such thermal power companies.

A deep reinforcement learning-based bidding strategy for participants in a peer-to-peer energy trading scenario

“…For instance, Wang et al (2019) formulated the sequential energy bidding problem as a Markov decision process (MDP) and utilized the classic Q-learning method to reflect personalized bidding preferences. Liu et al (2021) used the multi-agent deep deterministic policy gradient (MADDPG) algorithm to address the non-cooperative and cooperative energy trading games between power companies. Tao et al (2022) utilized the deep deterministic policy gradient (DDPG) method to generate the bidding price and bidding volume for an electric vehicle aggregator, which reduced the reliance of algorithm performance on the stochastic model.…”

Machine learning applications for electricity market agent-based models: A systematic literature review

“…They use the DQN reinforcement learning algorithm to maximise profit in peer-to-peer energy trading. Liu et al [42] also investigate the peerto-peer trading problem in a local energy market. They find that the community modelled are able to increase profitability through the use of DQN RL.…”

“…Market Type Application Algorithm Used 2021 Bose S. [6] Local energy market Peer to peer trading Roth-Erev 2021 Naseri N. [55] Local energy market Tariff design SARIMAX, MDP 2021 Tang C. [78] International/National Tariff design Novel WoLF-PHC 2021 Liu D. [42] International/National Bidding strategies, Peerto-Peer MADDPG 2021 Deng C. [12] International/National Demand response DDPG 2021 Viehmann J. [82] International/National Market investigation Q-Learning 2020 Tomin N. [80] Microgrid Electricity grid control Q-Learning 2020 Liang Y.…”

Dual-Sided Involvement of Energy Optimization and Strategic Bidding in Wind-PV System to Maximize Benefits for Customers and Power Providers

“…Furthermore, from the standpoint of power suppliers and consumers, strategic bidder faces difficulties, this category of transformation is also rationalized by the existing condition as such an unpreventable social assistance necessity [7,8]. Researchers develop bidding strategies for power provider organizations [10], facing challenges due to uncertain demand and rival actions, addressing various timelines [11]. In a perfect electricity market, power suppliers bid primarily on marginal cost.…”