Generalizing wave gestures from sparse examples for real-time character control

Abstract

Figure 1: Top: Skeletal, facial, and hand motions are tracked by off-the-shelf sensors. From sparse user-defined examples, our method reliably separates and extrapolates simultaneously-performed gestures to control characters in games or VR. Bottom left: Dog happy walk, neutral run, shaking, and sitting. Bottom right: Caterpillar crawl variations, controlled by varying amplitude, phase, and frequency of input gestures. AbstractMotion-tracked real-time character control is important for games and VR, but current solutions are limited: retargeting is hard for nonhuman characters, with locomotion bound to the sensing volume; and pose mappings are ambiguous with difficult dynamic motion control. We robustly estimate wave properties -amplitude, frequency, and phase -for a set of interactively-defined gestures by mapping user motions to a low-dimensional independent representation. The mapping separates simultaneous or intersecting gestures, and extrapolates gesture variations from single training examples. For animations such as locomotion, wave properties map naturally to stride length, step frequency, and progression, and allow smooth transitions from standing, to walking, to running. Interpolating outof-phase locomotions is hard, e.g., quadruped legs between walks and runs switch phase, so we introduce a new time-interpolation scheme to reduce artifacts. These improvements to real-time motiontracked character control are important for common cyclic animations. We validate this in a user study, and show versatility to apply to part-and full-body motions across a variety of sensors.

Inverse Kinematics Techniques in Computer Graphics: A Survey

“…When motion is smoothed through the convolution process, it interpolates fine details of the movement. Nevertheless, both learnt and deep learning methods seem currently to be among the most popular approaches for controlling human pose and reconstructing motion from sparse data [SH08, KTWZ10, LWC*11, RTK*15, XWCH15, HSKJ15] as well as for animating highly articulated human parts (e.g. hands) [dLGPF08, OKA11, LYTZ13].…”

“…[LWC*11] used MAP estimation of human poses in sequential mode to match the control signals obtained from motion sensors, increasing the temporal consistency of the movements. In addition, there are several methods that reconstruct motion streams from sparse representations by retrieving matched motion sequences from a motion capture library [SH08, RTK*15, XWCH15].…”

Style Invariant Locomotion Classification for Character Control

“…The human poses may or may not resemble the corresponding motion itself, so long as the user performs it consistently with the setup definition. The second method [RTK*15] classifies the actions and provides continuous amplitude, progress and frequency parameters through regression and local Fourier transformation. Here again, the input motion may or may not resemble the controlled movement, depending on how the system has been configured.…”

“…For progress stabilization, we use a filter akin to that proposed by Rhodin et al . [RTK*15]. This filter aims at stabilising the rate of change in progress, instead of smoothing the progress value directly.…”

“…The existence of such equipment has motivated research in action classification for character control, especially in gaming applications. One relevant aspect of character control through motion classification is the possibility of animating characters whose anatomic features are significantly different from those of the captured subject [SOL13, RTK*15].…”