A New Model-Based Method for Multi-View Human Body Tracking and Its Application to View Transfer in Image-Based Rendering

Liu, Zhong; Lin, Zhouchi; Wei, Xiguang; Chan, S.C.

doi:10.1109/tmm.2017.2767781

Cited by 19 publications

(6 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, we propose a complete set of keypoint detection, key point description, saliency region detection, and model alignment algorithms for human skeleton detection and extraction from local features. This paper mainly does the following work: (1) proposes to build a skeleton point detection model using an improved bottom-up scheme, which first detects all the skeleton point positions of the human body in the picture, and then reorganizes individual instances according to the association information, and the whole picture only needs to be entered once from the prediction network, thus eliminating the impact of uncertainty of the human body; (2) proposes to use the existing skeleton point and (2) propose to use the existing skeleton points to construct the association information between the skeleton points as a new feature to be provided to the CNN for training, so that the association information between the skeleton points can be obtained as a detection problem; and (3) use the multiscale equalization module to equalize the features of different scales separately and dynamically assign different attention weights to the features of different scales according to the loss function when detecting different joints, so that the features of different scales can be used more. The features at different scales are dynamically assigned different attention weights according to the loss function when detecting different nodes so that the features at different scales can be used more efficiently.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Three-Dimensional Diffusion Model in Sports Dance Video Human Skeleton Detection and Extraction

2021

Advances in Mathematical Physics

View full text Add to dashboard Cite

The research in this paper mainly includes as follows: for the principle of action recognition based on the 3D diffusion model convolutional neural network, the whole detection process is carried out from fine to coarse using a bottom-up approach; for the human skeleton detection accuracy, a multibranch multistage cascaded CNN structure is proposed, and this network structure enables the model to learn the relationship between the joints of the human body from the original image and effectively predict the occluded parts, allowing simultaneous prediction of skeleton point positions and skeleton point association information on the one hand, and refinement of the detection results in an iterative manner on the other. For the combination problem of discrete skeleton points, it is proposed to take the limb parts formed between skeleton points as information carriers, construct the skeleton point association information model using vector field, and consider it as a feature, to obtain the relationship between different skeleton points by using the detection method. It is pointed out that the reorganization problem of discrete skeleton points in multiperson scenes is an NP-Hard problem, which can be simplified by decomposing it into a set of subproblems of bipartite graph matching, thus proposing a matching algorithm for discrete skeleton points and optimizing it for the skeleton dislocation and algorithm problems of human occlusion. Compared with traditional two-dimensional images, audio, video, and other multimedia data, the 3D diffusion model data describe the 3D geometric morphological information of the target scene and are not affected by lighting changes, rotation, and scale transformation of the target and thus can describe the realistic scene more comprehensively and realistically. With the continuous updating of diffusion model acquisition equipment, the rapid development of 3D reconstruction technology, and the continuous enhancement of computing power, the research on the application of 3D diffusion model in the detection and extraction of a human skeleton in sports dance videos has become a hot direction in the field of computer vision and computer graphics. Among them, the feature detection description and model alignment of 3D nonrigid models are a fundamental problem with very important research value and significance and challenging at the same time, which has received wide attention from the academic community.

show abstract

Section: Discussionmentioning

confidence: 99%

“…The study of human skeleton detection in sports dance video images has been a very popular research direction in image processing and computer vision [2]. The human skeleton information can greatly help people analyze the behavior of the target human body in pictures or videos and lay the foundation for further processing of images and videos [3].…”

Section: Lessmentioning

confidence: 99%

Three-Dimensional Diffusion Model in Sports Dance Video Human Skeleton Detection and Extraction

2021

Advances in Mathematical Physics

View full text Add to dashboard Cite

show abstract

“…In the process of face recognition, the recognition defects caused by the changes in illumination, posture, face, etc., can only be compensated by three-dimensional face recognition technology. erefore, three-dimensional face recognition has become a relatively new research direction in the current image processing research [9]. e recognition method based on three-dimensional face data is mainly used to solve the difficult facial pose caused by the twodimensional face recognition method.…”

Section: Related Workmentioning

confidence: 99%

3D Face Modeling Algorithm for Film and Television Animation Based on Lightweight Convolutional Neural Network

Cheng

Peng

et al. 2021

Complexity

View full text Add to dashboard Cite

Through the analysis of facial feature extraction technology, this paper designs a lightweight convolutional neural network (LW-CNN). The LW-CNN model adopts a separable convolution structure, which can propose more accurate features with fewer parameters and can extract 3D feature points of a human face. In order to enhance the accuracy of feature extraction, a face detection method based on the inverted triangle structure is used to detect the face frame of the images in the training set before the model extracts the features. Aiming at the problem that the feature extraction algorithm based on the difference criterion cannot effectively extract the discriminative information, the Generalized Multiple Maximum Dispersion Difference Criterion (GMMSD) and the corresponding feature extraction algorithm are proposed. The algorithm uses the difference criterion instead of the entropy criterion to avoid the “small sample” problem, and the use of QR decomposition can extract more effective discriminative features for facial recognition, while also reducing the computational complexity of feature extraction. Compared with traditional feature extraction methods, GMMSD avoids the problem of “small samples” and does not require preprocessing steps on the samples; it uses QR decomposition to extract features from the original samples and retains the distribution characteristics of the original samples. According to different change matrices, GMMSD can evolve into different feature extraction algorithms, which shows the generalized characteristics of GMMSD. Experiments show that GMMSD can effectively extract facial identification features and improve the accuracy of facial recognition.

show abstract

“…Most of the methods [26], [27], [28], [29] on 3D pose estimation were originated from 2D pose estimation task. Some works [30], [31], [32], [33], [34], [35], [36] tried to develop multi-view based methods to get more accurate 3D pose estimation.…”

Section: Related Workmentioning

confidence: 99%

Wearable Motion Capture: Reconstructing and Predicting 3D Human Poses from Wearable Sensors

Moniruzzaman¹,

Yin²,

Hossain³

et al. 2021

Preprint

View full text Add to dashboard Cite

Identifying 3D human walking poses in unconstrained environments has many applications such as enabling prosthetists and clinicians to access the amputees’ walking functions outside clinics and helping amputees obtain an optimal walking condition with predictive control. Thus, we propose the wearable motion capture problem of reconstructing and predicting 3D human poses from the wearable IMU sensors and wearable cameras. To solve this challenging problem, we introduce a novel Attention-Oriented Recurrent Neural Network (AttRNet) that contains a sensor-wise attention-oriented recurrent encoder, a reconstruction module, and a dynamic temporal attention-oriented recurrent decoder, to reconstruct the current pose and predict the future poses. To evaluate our approach, we collected a new WearableMotionCapture dataset using wearable IMUs and wearable video cameras, along with the musculoskeletal joint angle ground truth. The proposed AttRNet shows high accuracy on theWearableMotionCapture dataset, and it also outperforms the current best methods on two public pose prediction datasets with IMU-only data: DIP-IMU and TotalCaputre. The source codes and the new dataset will be publicly available on https://github.com/MoniruzzamanMd/Wearable-Motion-Capture.<br>

show abstract

A New Model-Based Method for Multi-View Human Body Tracking and Its Application to View Transfer in Image-Based Rendering

Cited by 19 publications

References 51 publications

Three-Dimensional Diffusion Model in Sports Dance Video Human Skeleton Detection and Extraction

Three-Dimensional Diffusion Model in Sports Dance Video Human Skeleton Detection and Extraction

3D Face Modeling Algorithm for Film and Television Animation Based on Lightweight Convolutional Neural Network

Wearable Motion Capture: Reconstructing and Predicting 3D Human Poses from Wearable Sensors

Contact Info

Product

Resources

About