Scenario-assisted Deep Reinforcement Learning

Constrained Reinforcement Learning for Robotics via Scenario-Based Programming

“…In [64], the authors proposed an integration between SBP and DRL training, using a reward shaping approach that penalizes the agent when rules are violated. show a comparison of our approach with that of [64]: graph (a) compares the success rates of the two approaches with all three scenario-based rules, and graph (b) compares the frequency of violations to the avoid back-and-forth rotation rule. For these two experiments, we configured the approach of [64] to use fixed penalties of 1.0 and 0.05, respectively.…”

“…A Rule-Based Approach. Following the approach of [64], we integrated a scenario-based program into the DRL training process, in order to remove the aforementioned undesirable behaviors. More concretely, we created specific scenarios to rule out each of the three aforementioned undesirable behaviors we observed.…”

“…These events allow the various scenarios to keep track and react to the agent's actions. Similarly to [64], we refer to these e at events as external events, indicating that they can only be triggered when requested from outside the SB program proper.…”

“…By convention, we assume that after each triggering of a single, external event, the scenario-based program executes a sequence of internal events (a super-step [64]), until it returns to a steady-state and then waits for another external event.…”

“…The novelty of our approach, compared to [64], is in the strategy by which we use scenarios to affect the training process. Specifically, we define the DRL cost function to correspond to violations of scenario constraints by the DRL agent.…”