Bidirectional recurrent autoencoder for 3D skeleton motion data refinement

Li, Shujie; Zhou, Yang; Zhu, Hai; Xie, Wei; Zhao, Yang; Liu, Xiaoping

doi:10.1016/j.cag.2019.03.010

Cited by 33 publications

(52 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Data‐driven methods have been proposed to preserve the spatio‐temporal features of highly coordinated human motions [HJ10, FJX*14, XFJ*15]. Recently, deep learning frameworks have shown a high‐quality performance in the correction of motion data [HSKJ15, HSK16, Hol18, MLCC17, LZZ*19]. While the direction that the deep learning framework pursues is promising, its applicability in practice can be limited as it requires training priors, such as noisy or corrupted motion data; the resulting system is inevitably specialized to handle the situations defined by the given data.…”

Section: Related Workmentioning

confidence: 99%

Synthesizing Character Animation with Smoothly Decomposed Motion Layers

Eom

Choi

Cho

et al. 2019

Computer Graphics Forum

View full text Add to dashboard Cite

The processing of captured motion is an essential task for undertaking the synthesis of high‐quality character animation. The motion decomposition techniques investigated in prior work extract meaningful motion primitives that help to facilitate this process. Carefully selected motion primitives can play a major role in various motion‐synthesis tasks, such as interpolation, blending, warping, editing or the generation of new motions. Unfortunately, for a complex character motion, finding generic motion primitives by decomposition is an intractable problem due to the compound nature of the behaviours of such characters. Additionally, decomposed motion primitives tend to be too limited for the chosen model to cover a broad range of motion‐synthesis tasks. To address these challenges, we propose a generative motion decomposition framework in which the decomposed motion primitives are applicable to a wide range of motion‐synthesis tasks. Technically, the input motion is smoothly decomposed into three motion layers. These are base‐level motion, a layer with controllable motion displacements and a layer with high‐frequency residuals. The final motion can easily be synthesized simply by changing a single user parameter that is linked to the layer of controllable motion displacements or by imposing suitable temporal correspondences to the decomposition framework. Our experiments show that this decomposition provides a great deal of flexibility in several motion synthesis scenarios: denoising, style modulation, upsampling and time warping.

show abstract

Section: Related Workmentioning

confidence: 99%

Synthesizing Character Animation with Smoothly Decomposed Motion Layers

Eom

Choi

Cho

et al. 2019

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…Thus, these researchers trained an encoder-bidirectional-filter (EBF) network to postprocess EBD results. Holden [12] also used a smoothing step to filter jittery movements, but postprocessing steps are time-consuming and not suitable for real-time [13] in which malformed or unnatural parts are squared. (e) Pose refined by the proposed BRA-P.…”

Section: Introductionmentioning

confidence: 99%

“…motion data acquisition systems. Li et al [13] proposed a bidirectional recurrent autoencoder that can improve the kinematic information expression ability of the network by imposing smoothness and bone-length constraints. However, unfortunately, smoothness and bone-length constraints cannot satisfactorily maintain the kinematic information, and the noisy data and target clean data have different skeleton topologies.…”

Section: Introductionmentioning

confidence: 99%

“…In line with [13], we also use the bidirectional long shortterm memory recurrent neural network (B-LSTM-RNN) architecture [14], [15] to refine noisy motion data. Our previous work in [13] noted that the refinement network based on the B-LSTM-RNN architecture does not require noise amplitude as a priori knowledge. In this paper, we found that the B-LSTM-RNN architecture network also does not require the noise type as prior knowledge.…”

Section: Introductionmentioning

confidence: 99%

“…Our strategy is to train a denoising autoencoder consisting of E d and D d by noisy-clean motion pairs X N and X C based on a pretrained perceptual autoencoder trained by clean motion data and consisting of E p and D p . The loss function for the perceptual autoencoder consists of three elements, i.e., reproduction loss, smooth loss and bone-length loss, which are the same as the losses used in [13]. In contrast, the loss function for training the denoising autoencoder consists of four elements: perceptual loss, reproduction loss, smooth loss and bone-length loss.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Perceptual-Based Noise-Agnostic 3D Skeleton Motion Data Refinement Network

Zhu

Zheng

et al. 2020

IEEE Access

View full text Add to dashboard Cite

In this paper, we demonstrate a perceptual-based 3D skeleton motion data refinement method based on a bidirectional recurrent autoencoder, called BRA-P. Three main technical contributions are made by the proposed network. First, the proposed BRA-P can address noisy data with different noise types and amplitudes using one network, and this attribute makes the approach more suitable for raw motion data with heterogeneous mixed noise. Second, due to the usage of perceptual loss, which measures the difference in high-level features extracted by a pretrained perceptual autoencoder, BRA-P improves the perceptual similarity between refined motion data and clean motion data, especially for the case where the noisy data and target clean data have different topologies. Third, BRA-P further improves the bone-length consistency and smoothness of the refined motion using the perceptual autoencoder as a postprocessing network. Ablation experiments verify the effect of the three technical contributions of our approach. The results of the experiments on synthetic noise data and raw motion data captured by Kinect demonstrate that our method outperforms several state-of-the-art methods in the cleaning of mixed-noise data by one network.

show abstract