Deep learning based six‐dimensional pose estimation in virtual reality

Abstract

Virtual reality technology, with its continuous development, is gradually applied to healthcare, education, business, and other fields. In the application of the technology, position and attitude estimation, as a space positioning technology, is indispensable. Traditional pose estimation has the problems of high dependence on environment and great complexity. But convolutional neural network (CNN) and other technologies with computational intelligence provide a strong guarantee for the progress of pose estimation. This article, based on the theory of CNN in deep learning, as well as monocular vision system and target sample set with markers, proposes a method for estimation of target position and attitude, and at the same time, describes in detail a general way of making dataset with markers based on simulation environment. In this article, the comparative experiments of different network structures show that this measurement method can avoid manual extraction of complex image features, and realize fast, arbitrary and accurate measurement, which plays a key role in pose and attitude measurement. Moreover, the visual correspondence between the world coordinate system and the pixel coordinate system is proved effectively by quaternion.

DOPE++: 6D pose estimation algorithm for weakly textured objects based on deep neural networks

“…However, the lack of real data for training the network makes it difficult to expand the network to new application scenarios, such as in the field of smart manufacturing [ 32 – 34 ] and autonomous driving [ 35 , 36 ]. For this purpose, we use virtual reality techniques [ 37 – 40 ] to produce datasets on weakly textured industrial parts. We independently design a series of comparative experiments to verify the advantages of using virtual reality technology to produce datasets, such as avoiding the problems of a single background, small changes in object position and pose, and easy overfitting that exist in real datasets of YCB videos [ 41 , 42 ].…”

Uncertainty prediction in human pose estimation based on Gaussian heatmap

“…As a cornerstone of computer vision, 2D HPE drives the prosperity and development of action recognition [1], pedestrian tracking [2], gesture recognition [3], gait recognition [4] and other related fields [5], [6]. Meanwhile, real-time 2D HPE extends its influence to daily activity scenarios, including intelligent video surveillance [7], patient monitoring systems [8], virtual reality [9], autonomous drive [10], human animation [11], smart home [12], [13], athlete-assisted training [14], etc.…”