Somatosensory interaction for real-time large scale roaming

Abstract

Real-time somatosensory roaming in large-scale scene is a challenging problem because of great computation for the scene rendering, the real-time posture recognition and the instant response to the user's action. We present a novel somatosensory interactive roaming method in a realistic large-scale forest scene based on Microsoft Kinect. First, we construct a realistic large-scale forest scene based on GPU acceleration. Then for each action of the user, we encode the directions between joints into direction codes, and generate a posture vector which is used to compare with our posture template table to achieve posture recognition. Combining our forest scene with our multi-thread posture recognition, we can somatosensory interactively control roaming direction, velocity, and changes of light and shadow in real-time for the scene with millions of triangles. Our posture recognition algorithm is very fast and can response to the change of the user's posture in time. Experiments demonstrate that our method could successfully roam in the large-scale scene with a natural manner, which provides a feasible solution for the challenging problem.