Real-time labeling of non-rigid motion capture marker sets

Abstract

Passive optical motion capture is one of the predominant technologies for capturing high fidelity human motion, and is a workhorse in a large number of areas such as bio-mechanics, film and video games. While most state-of-the-art systems can automatically identify and track markers on the larger parts of the human body, the markers attached to the fingers and face provide unique challenges and usually require extensive manual cleanup. In this work we present a robust online method for identification and tracking of passive motion capture markers attached to non-rigid structures. The method is especially suited for large capture volumes and sparse marker sets. Once trained, our system can automatically initialize and track the markers, and the subject may exit and enter the capture volume at will. By using multiple assignment hypotheses and soft decisions, it can robustly recover from a difficult situation with many simultaneous occlusions and false observations (ghost markers). In three experiments, we evaluate the method for labeling a variety of marker configurations for finger and facial capture. We also compare the results with two of the most widely used motion capture platforms: Motion Analysis Cortex and Vicon Blade. The results show that our method is better at attaining correct marker labels and is especially beneficial for real-time applications.<p>QC 20171127</p

Online optical marker-based hand tracking with deep labels

“…We compare our real-time tracking results with Alexanderson et al [2017] against ground truth on the four captured test sequences (Table 5). Since Alexanderson et al [2017] is designed for a sparse marker set requiring three markers on the back of the hand already labeled, it doesn't perform well on a dense marker set with a considerable amount of ghost markers or any of the three back markers is occluded. We also tried to compare with the commercial system Vicon [2018].…”

“…Recent work has sought to reduce the marker density on a hand and optimize the marker layout, to disambiguate labels and minimize the impact on pose reconstruction [Alexanderson et al 2017;Schröder et al 2015;Wheatland et al 2013]. However, methods using sparse marker sets rely on priors to hallucinate missing information about finger motion.…”

Virtual hands in VR

“…Even with a reduced marker set, labeling each marker correctly remains a challenging task that usually requires tedious manual effort. Alexanderson et al [1,2] proposed an automatic marker labeling algorithm using multi-hypothesis tracking. They use Gaussian Mixture Models (GMMs) to represent the spatial distribution of marker positions local to the hand.…”

Hand Rehabilitation and Telemonitoring through Smart Toys

“…A possible approach is based on using cameras to survey movement. Motion capture systems, based on using passive markers to robustly identify the finger joints on the camera images, were first proposed [28]. Although this approach is potentially accurate, it is limited to the laboratory domain, as the set-up time required to attach the markers to the hand makes this approach unfeasible at home.…”