Towards Ecosystem Management from Greedy Reinforcement Learning in a Predator-Prey Setting

A Sustainable Ecosystem through Emergent Cooperation in Multi-Agent Reinforcement Learning

“…Assuming predators are generally selfish, this paper aims for cooperative behavior beyond overcoming the tragedy of the commons in form of group hunting and analyzes which factors impact such cooperation. Prior work (Leibo et al, 2017;Pérolat et al, 2017;Ritz et al, 2020) does not consider predator starvation, which we additionally regard in this paper. Apart from splitting the total reward for cooperatively caught prey, there is no reward shaping.…”

“…Regarding self-regulation, prior work of Ritz et al (2020) only considered a single predator. This paper assumes a group of non reproducing predators under starvation pressure and analyzes if collective self-regulation, i.e.…”

“…They showed that a single RL predator can learn to maintain a herd of prey in absence of starvation for long-term benefit. This paper extends the scenario of Ritz et al (2020) to multiple RL predators and further adds starvation, putting pressure on the predators. According to game theory, this might turn into a tragedy of the commons if selfish individuals decide to greedily hunt down the prey population before their conspecifics do, ultimately leading to extinction prey and predators.…”

“…An extensive survey on using RL for sustainable energy systems has been done by Yang et al (2020), showing a broad range of applications such as integrating renewable energy with their implied uncertainty into energy networks while optimizing economic objectives, energy utilization and environmental impacts. Recently, Ritz et al (2020) analyzed a predatorprey environment in terms of sustainable behavior. They showed that a single RL predator can learn to maintain a herd of prey in absence of starvation for long-term benefit.…”

Co-Evolution of Predator-Prey Ecosystems by Reinforcement Learning Agents

“…For these individuals’ objectives, a multi-agent reinforcement learning (MARL) method based on Deep Q-Networks (DQN) is utilized to execute a foraging task. Ritz et al [ 30 ] applied reinforcement learning to train a predator in a single predator and a multi prey system, in which a predator evolves by means of RL and interacts with preys, which are non-RL agents. In their works, a version of DQN was utilized for the RL framework with long-term reward discounting and stacked observations.…”