SuperSuit: Simple Microwrappers for Reinforcement Learning Environments

Terry, Justin K.; Black, Benjamin; Hari, Ananth

doi:10.48550/arxiv.2008.08932

Cited by 5 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The implementation of one RL agent with agentenvironment interaction is a standard functionality of OpenAI gym [24]. To add multiple independent agents to the same environment we use a combination of the PettingZoo [25] and SuperSuit [26] libraries. PettingZoo is a multi-agent reinforcement learning wrapper that combines multiple agents' actions before passing them to the OpenAI gym environment (which takes just one action argument); SuperSuit provides pre-processing of the environment and allows for agents in the grid environment to have a non-uniform actionspace as dictated by the number of available energy systems in their buildings.…”

Section: Multi-agent Reinforcement Learning Frameworkmentioning

confidence: 99%

GridLearn: Multiagent Reinforcement Learning for Grid-Aware Building Energy Management

Pigott¹,

Crozier²,

Baker³

et al. 2021

Preprint

View full text Add to dashboard Cite

Increasing amounts of distributed generation in distribution networks can provide both challenges and opportunities for voltage regulation across the network. Intelligent control of smart inverters and other smart building energy management systems can be leveraged to alleviate these issues. GridLearn is a multiagent reinforcement learning platform that incorporates both building energy models and power flow models to achieve grid level goals, by controlling behind-the-meter resources. This study demonstrates how multi-agent reinforcement learning can preserve building owner privacy and comfort while pursuing grid-level objectives. Building upon the CityLearn framework which considers RL for building-level goals, this work expands the framework to a network setting where grid-level goals are additionally considered. As a case study, we consider voltage regulation on the IEEE-33 bus network using controllable building loads, energy storage, and smart inverters. The results show that the RL agents nominally reduce instances of undervoltages and reduce instances of overvoltages by 34%.

show abstract

Section: Multi-agent Reinforcement Learning Frameworkmentioning

confidence: 99%

GridLearn: Multiagent Reinforcement Learning for Grid-Aware Building Energy Management

Pigott¹,

Crozier²,

Baker³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The slightly different API used by dm_control means that fewer algorithm libraries support its use or do so through the use of wrappers which enforce conformity with the Gym API. SuperSuit [13] is an integration tool between environments and algorithms that implements a number of more complex wrappers to Gym environments, such as frame stacking or delaying observations. All of these wrappers use the Gym or Gymnasium API and their functions can be or are replicated in CoRL.…”

Section: Introductionmentioning

confidence: 99%

CoRL: Environment Creation and Management Focused on System Integration

Merrick¹,

K.²,

Long³

et al. 2023

Preprint

View full text Add to dashboard Cite

Existing reinforcement learning environment libraries use monolithic environment classes, provide shallow methods for altering agent observation and action spaces, and/or are tied to a specific simulation environment. The Core Reinforcement Learning library (CoRL) is a modular, composable, and hyper-configurable environment creation tool. It allows minute control over agent observations, rewards, and done conditions through the use of easy-to-read configuration files, pydantic [1] validators, and a functor design pattern. Using integration pathways allows agents to be quickly implemented in new simulation environments, encourages rapid exploration, and enables transition of knowledge from low-fidelity to high-fidelity simulations. Natively multi-agent design and integration with Ray/RLLib [2] at release allow for easy scalability of agent complexity and computing power. The code is publicly released and available at https://github.com/act3-ace/CoRL.

show abstract

“…e) Implementation: 1 We used the proximal policy optimization algorithm [15] of Stable-Baselines3 2.1.0 [16] to build our RL agent, using its default hyper-parameters. The multiagent RL auction environment was realized using Gymnasium 0.29.0 [17] and PettingZoo 1.24.1 [18] combined with Supersuit 3.9.0 [19] to create a vector environment for multi-agent training. In such environments, it is necessary to normalize the continuous state and observation spaces to finite ranges.…”

mentioning

confidence: 99%

Gambling on Reconfigurable Intelligent Surfaces

Schwarz

2024

IEEE Commun. Lett.

View full text Add to dashboard Cite

We consider multi-operator wireless networks where broadband reconfigurable intelligent surfaces (RISs) effectively cover the transmission bands of all operators. These RISs are supplied by a dedicated provider and dynamically leased ondemand to individual operators to support their transmissions. When an operator takes control of a RIS, it can adjust its phaseresponse to meet the requirements of its users. This sets the stage for a competitive scenario where operators vie for control of RISs. To address this competition, we introduce an auction format designed to efficiently allocate RISs to operators. Furthermore, we develop a multi-agent reinforcement learning environment to optimize operators' bidding strategies, demonstrating its superiority over the heuristic dominant strategy of greedy bidding.

show abstract

SuperSuit: Simple Microwrappers for Reinforcement Learning Environments

Cited by 5 publications

References 7 publications

GridLearn: Multiagent Reinforcement Learning for Grid-Aware Building Energy Management

GridLearn: Multiagent Reinforcement Learning for Grid-Aware Building Energy Management

CoRL: Environment Creation and Management Focused on System Integration

Gambling on Reconfigurable Intelligent Surfaces

Contact Info

Product

Resources

About