Carl

Abstract

Motion synthesis in a dynamic environment has been a long-standing problem for character animation. Methods using motion capture data tend to scale poorly in complex environments because of their larger capturing and labeling requirement. Physics-based controllers are effective in this regard, albeit less controllable. In this paper, we present CARL, a quadruped agent that can be controlled with high-level directives and react naturally to dynamic environments. Starting with an agent that can imitate individual animation clips, we use Generative Adversarial Networks to adapt high-level controls, such as speed and heading, to action distributions that correspond to the original animations. Further fine-tuning through the deep reinforcement learning enables the agent to recover from unseen external perturbations while producing smooth transitions. It then becomes straightforward to create autonomous agents in dynamic environments by adding navigation modules over the entire process. We evaluate our approach by measuring the agent's ability to follow user control and provide a visual analysis of the generated motion to show its effectiveness.

Neural3Points: Learning to Generate Physically Realistic Full‐body Motion for Virtual Reality Users

“…To create a multi‐skilled character, individual controllers can be organized and scheduled by high‐level policies [PBYv17; LH17; MAP*19] or be used to train an integrated policy [MHG*19; MTA*20; WGH20]. Direct training of multi‐skilled policies can also be achieved using a mixture of expert structure [PCZ*19; LSCC20] or with the help of adversarial losses [MTT*17]. Combining the advantage of both data‐driven motion generators and RL‐based control policy is another avenue to creating multi‐skilled and interactive control policies [PRL*19; BCHF19], and similar ideas are also adopted to reduce the ambiguity caused by incomplete input signals [YPL21; SGXT20; XWI*21].…”

Amp

“…However, since the realism of the character's motions is enforced implicitly through the latent representation, rather than explicitly through an objective function, it is still possible for the high-level controller to specify latent encodings that produce unnatural behaviors [Merel et al 2020;Peng et al 2019a]. Luo et al [2020] proposed an adversarial domain confusion loss to prevent the high-level controller from specifying encodings that are different from those observed during pre-training. However, since this adversarial objective is applied in the latent space, rather than on the actual motions produced by the character, the model is nonetheless prone to generating unnatural behaviors.…”

“…But when the target is further away, the character automatically transitions into a run. These intricate behaviors arise naturally from the motion prior, without requiring a motion planner to explicitly select which motion the character should execute in a given scenario, such as those used in prior systems [Bergamin et al 2019;Luo et al 2020;Peng et al 2017]. In addition to standard locomotion gaits, the motion prior can also be trained for more stylistic behaviors, such as walking like a shambling zombie or walking in a stealthy manner.…”

“…Both AMP and the latent space models are able to produce substantially more life-like behaviors than the baseline models. For the latent space models, since the low-level and high-level controllers are trained separately, it is possible for the distribution of encodings specified by the high-level controller to be different than the distribution of encodings observed by the low-level controller during pre-training [Luo et al 2020]. This in turn can result in unnatural motions that deviate from the behaviors depicted in the original dataset.…”

SPNets: Human-like Navigation Behaviors with Uncertain Goals

“…Approaches for navigation can either be supervised with known training data (as in [Pfeiffer et al 2017]), or the network can be trained to optimize path cost using reinforcement learning (as in [Tamar et al 2016] and [2018]). Deep reinforcement learning (DRL) has recently shown promise for navigation as a fine-tuning final step, as was done in [Pfeiffer et al 2018] and [Luo et al 2020]. Very recent work has sought to improve the practicality of DRL navigation by augmenting it with classical, analytical planners ( [Chaplot et al 2020]).…”