Local motion phases for learning multi-contact character movements

Abstract

Training a bipedal character to play basketball and interact with objects, or a quadruped character to move in various locomotion modes, are difficult tasks due to the fast and complex contacts happening during the motion. In this paper, we propose a novel framework to learn fast and dynamic character interactions that involve multiple contacts between the body and an object, another character and the environment, from a rich, unstructured motion capture database. We use one-on-one basketball play and character interactions with the environment as examples. To achieve this task, we propose a novel feature called local motion phase, that can help neural networks to learn asynchronous movements of each bone and its interaction with external objects such as a ball or an environment. We also propose a novel generative scheme to reproduce a wide variation of movements from abstract control signals given by a gamepad, which can be useful for changing the style of the motion under the same context. Our scheme is useful for animating contact-rich, complex interactions for real-time applications such as computer games.

Adoption of Machine Learning Algorithm-Based Intelligent Basketball Training Robot in Athlete Injury Prevention

“…In the next step of feature extraction, the unit action is analyzed, and the attribute features are extracted and taken as samples. At the final classifier stage, the sample data are constructed into a classification model according to different classification principles to complete the classification of the sample data (Mejia-Ruda et al, 2018 ; Mullard, 2019 ; Starke et al, 2020 ).…”

Creating Deep Learning-based Acrobatic Videos Using Imitation Videos

“…Eom et al [33] created a physics-based character that can effectively catch a flying object by using the model predictive control framework that synthesizes the movements of the eyes and head. Starke et al [34] proposed a effective method to generate motion in ball games handling an object by training local bones individually. Especially, it can generate basketball dribbling naturally and various connected motions.…”

MoGlow

“…Proposals to overcome this issue in deterministic models include learning and predicting foot contacts [Holden et al 2016], or the phase or pace [Pavllo et al 2018] of the gait cycle. Starke et al [2020] generalised the idea of motion phase to complex motion by letting each bone in a character follow a separate motion phase. Autoregressively feeding in previouslygenerated poses might help combat regression to the mean, and has been used in motion generation without control inputs [Bütepage et al 2017;Fragkiadaki et al 2015;Zhou et al 2018].…”

“…Variations of GANs [Sadoughi and Busso 2018] and adversarial training [Ferstl et al 2019;Starke et al 2020;] have also been applied for motion generation and the related task of generating speech-driven video of talking faces [Pham et al 2018;Pumarola et al 2018;Vougioukas et al 2018Vougioukas et al , 2020. In contrast to GANs and VAEs, Starke et al [2020] add latent-space noise to motion only at synthesis time (not during training), to obtain more varied motion, albeit at the expense of deviating from the desired input control. This approach also means that the distribution of the motion is not learned, and need not match that of natural motion.…”