The interaction of multiple autonomous agents gives rise to highly dynamic and nondeterministic environments, contributing to the complexity in applications such as automated financial markets, smart grids, or robotics. Due to the sheer number of situations that may arise, it is not possible to foresee and program the optimal behaviour for all agents beforehand. Consequently, it becomes essential for the success of the system that the agents can learn their optimal behaviour and adapt to new situations or circumstances. The past two decades have seen the emergence of reinforcement learning, both in single and multi-agent settings, as a strong, robust and adaptive learning paradigm. Progress has been substantial, and a wide range of algorithms are now available. An important challenge in the domain of multi-agent learning is to gain qualitative insights into the resulting system dynamics. In the past decade, tools and methods from evolutionary game theory have been successfully employed to study multi-agent learning dynamics formally in strategic interactions. This article surveys the dynamical models that have been derived for various multi-agent reinforcement learning algorithms, making it possible to study and compare them qualitatively. Furthermore, new learning algorithms that have been introduced using these evolutionary game theoretic tools are reviewed. The evolutionary models can be used to study complex strategic interactions. Examples of such analysis are given for the domains of automated trading in stock markets and collision avoidance in multi-robot systems. The paper provides a roadmap on the progress that has been achieved in analysing the evolutionary dynamics of multi-agent learning by highlighting the main results and accomplishments.
Triggered by the increased fluctuations of renewable energy sources, the European Commission stated the need for integrated short-term energy markets (e.g., intraday), and recognized the facilitating role that local energy communities could play. In particular, microgrids and energy communities are expected to play a crucial part in guaranteeing the balance between generation and consumption on a local level. Local energy markets empower small players and provide a stepping stone towards fully transactive energy systems. In this paper we evaluate such a fully integrated transactive system by (1) modelling the energy resource management problem of a microgrid under uncertainty considering flexible loads and market participation (solved via two-stage stochastic programming), (2) modelling a wholesale market and a local market, and (3) coupling these elements into an integrated transactive energy simulation. Results under a realistic case study (varying prices and competitiveness of local markets) show the effectiveness of the transactive system resulting in a reduction of up to 75% of the expected costs when local markets and flexibility are considered. This illustrates how local markets can facilitate the trade of energy, thereby increasing the tolerable penetration of renewable resources and facilitating the energy transition.Index Terms-Demand response, local electricity markets, microgrids, transactive energy, smart grids, stochastic optimization. NOTATIONIndices: e energy storage systems (ESSs) i distributed generation (DG) units l, m, s, t, v loads, markets, scenarios, periods, electric vehicles (EVs) Sets and subsets: Ω DG , Ω load set of DG units/loads Ω d DG ,Ω nd DG subset of dispatchable/non-dispatchable DG units Ω curt load ,Ω inte load subset of curtailable/interruptible loads Ω shift load subset of shiftable loads Parameters: C DG generation cost of DG unit (m.u./kWh) C ESS − ,C EV − discharging cost of ESS/EV (m.u./kWh) Ccurt,C inte ,C shift load curtailment/interruption/shift cost (m.u./kWh) C imb grid imbalance cost (m.u./kWh) M P electricity market price (m.u./kWh) Ne, N i , N l number of ESS/DG/loads Nm, Ns, Nv number of markets/scenarios/EVs Pcurt max maximum load reduction of Ω curt load (kW) P DG max/min maximum/minimum power of dispatchable DGs (kW) P DG nd forecast power of non-dispatchable DGs (kW) P ESS/EV + max maximum charge rate of ESSs/EV (kW) P ESS/EV − max maximum discharge rate of ESSs/EV (kW) P ESS max/min maximum/minimum energy capacity of ESSs (kWh) P EV max/min maximum/minimum energy capacity of EVs (kWh) P EV trip forecasted energy demand for EVs' trip (kWh) P load forecasted active power of loads (kW) P offer max/min maximum/minimum energy offer in markets (kW) P shift forecasted power of Ω shift load in T shift (kW) P shift max maximum load shifted of Ω shift load in T shift (kW) T number of periods T shift shift interval of Ω shift load T start shift /T end shift earliest/latest possible period for load shift of Ω shift load η EV + /EV − charging/discharging efficiency of EV...
This paper presents an automated peer-to-peer negotiation strategy for settling energy contracts among prosumers in a Residential Energy Cooperative considering heterogeneity prosumer preferences. The heterogeneity arises from prosumers' evaluation of energy contracts through multiple societal and environmental criteria and the prosumers' private preferences over those criteria. The prosumers engage in bilateral negotiations with peers to mutually agree on periodical energy contracts/loans consisting of the energy volume to be exchanged at that period and the return time of the exchanged energy. The negotiating prosumers navigate through a common negotiation domain consisting of potential energy contracts and evaluate those contracts from their valuations on the entailed criteria against a utility function that is robust against generation and demand uncertainty. From the repeated interactions, a prosumer gradually learns about the compatibility of its peers in reaching energy contracts that are closer to Nash solutions. Empirical evaluation on real demand, generation and storage profiles -in multiple system scales -illustrates that the proposed negotiation based strategy can increase the system efficiency (measured by utilitarian social welfare) and fairness (measured by Nash social welfare) over a baseline strategy and an individual flexibility control strategy representing the status quo strategy. We thus elicit system benefits from peer-to-peer flexibility exchange already without any central coordination and market operator, providing a simple yet flexible and effective paradigm that complements existing markets.
Computers that negotiate on our behalf hold great promise for the future and will even become indispensable in emerging application domains such as the smart grid and the Internet of Things. Much research has thus been expended to create agents that are able to negotiate in an abundance of circumstances. However, up until now, truly autonomous negotiators have rarely been deployed in real-world applications. This paper sizes up current negotiating agents and explores a number of technological, societal and ethical challenges that autonomous negotiation systems have brought about. The questions we address are: in what sense are these systems autonomous, what has been holding back their further proliferation, and is their spread something we should encourage? We relate the automated negotiation research agenda to dimensions of autonomy and distill three major themes that we believe will propel autonomous negotiation forward: accurate representation, long-term perspective, and user trust. We argue these orthogonal research directions need to be aligned and advanced in unison to sustain tangible progress in the field.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.