Improving Reinforcement Learning with Human Assistance: An Argument for Human Subject Studies with HIPPO Gym

Abstract

Reinforcement learning (RL) is a popular machine learning paradigm for game playing, robotics control, and other sequential decision tasks. However, RL agents often have long learning times with high data requirements because they begin by acting randomly. In order to better learn in complex tasks, this article argues that an external teacher can often significantly help the RL agent learn.OpenAI Gym is a common framework for RL research, including a large number of standard environments and agents, making RL research significantly more accessible. This article introduces our new open-source RL framework, the Human Input Parsing Platform for Openai Gym (HIPPO Gym), and the design decisions that went into its creation. The goal of this platform is to facilitate human-RL research, again lowering the bar so that more researchers can quickly investigate different ways that human teachers could assist RL agents, including learning from demonstrations, learning from feedback, or curriculum learning.

VISITOR: Visual Interactive State Sequence Exploration for Reinforcement Learning

“…This study showed that an information-gain objective decreasing uncertainty was well suited to propose effective questions. There has been a recent effort to provide testing environments for human-subject experimentation in reinforcement learning (Taylor et al, 2021) or active querying (Bıyık et al, 2022b). However, existing work has focused on simple feedback types and user interfaces, and more extensive human studies as possible future research directions have been highlighted.…”