Hybrid system identification using switching density networks

Abstract

Behaviour cloning is a commonly used strategy for imitation learning and can be extremely effective in constrained domains. However, in cases where the dynamics of an environment may be state dependent and varying, behaviour cloning places a burden on model capacity and the number of demonstrations required. This paper introduces switching density networks, which rely on a categorical reparametrisation for hybrid system identification. This results in a network comprising a classification layer that is followed by a regression layer. We use switching density networks to predict the parameters of hybrid control laws, which are toggled by a switching layer to produce different controller outputs, when conditioned on an input state. This work shows how switching density networks can be used for hybrid system identification in a variety of tasks, successfully identifying the key joint angle goals that make up manipulation tasks, while simultaneously learning image-based goal classifiers and regression networks that predict joint angles from images. We also show that they can cluster the phase space of an inverted pendulum, identifying the balance, spin and pump controllers required to solve this task. Switching density networks can be difficult to train, but we introduce a cross entropy regularisation loss that stabilises training.

Hierarchical Decomposition of Nonlinear Dynamics and Control for System Identification and Policy Distillation

“…Switching systems have also served as a powerful tool in various imitation learning approaches. Calinon et al ( 2010) combine traditional HMMs with Gaussian mixture regression to represent trajectory distributions, while Daniel et al (2016) use a hidden semi-Markov model to learn hierarchical policies and Burke et al (2019) introduced switching density networks for system identification and behavioral cloning. Finally, excellent work on hierarchical decomposition of policies in a fully Bayesian framework is introduced by Šošić et al (2017), albeit under known transition dynamics.…”

Learning behavioral models by recurrent neural networks with discrete latent representations with application to a flexible industrial conveyor

“…The extraction of the policy implemented by a controller (being human or automatic) is often referred to as behavior cloning within the machine learning community, and we inherit here the same terminology. Specifically, the scope of behavior cloning is to learn a policy by imitation, i.e., the action to be performed in a given system state by extrapolating experience from a set of observationaction sequences [40]. With reference to system identification, the target is to imitate the behavior performed by an instructor (e.g., an automated control system or advanced logics on supervisory systems) by observing it operating in closed-loop over the controlled system (e.g., a device, a machine or a process).…”