Interaction capture using magnetic sensors

Abstract

Capturing a close interaction between an actor and an object can be difficult as a result of occlusion and having to recreate the geometry of the scene accurately. In this paper, we propose a technique that allows us to capture the object's motion and geometry alongside the actor's movements and optionally the local environment, using a magnetic motion capture system and an RGB-D sensor. This not only gives greater information when placing a character in a scene but enables us to digitally recreate the scene in motion without significant animator work after capture. The use of magnetic sensors prevents occlusion or marker confusion that is common in optical techniques when dealing with close interactions, as the magnetic sensors do not require direct line of sight to a camera. The geometry reconstruction ensures that the proportions of objects and surfaces the character interacts with are accurate and alleviates the need for an artist to model the object. We perform validation of the results by comparison with an optical system and show a variety of motions, such as using a screwdriver or removing a cap to drink from a bottle, that can be captured using our technique.

Reconstruction of Dexterous 3D Motion Data from a Flexible Magnetic Sensor with Deep Learning and Structure-Aware Filtering

“…These markers are also too large. To sense magnetic signals with stronger intensity, electromagnetic markers can be used with wires or embedded batteries [6,30,32]. Such systems can accurately track markers over a larger range, but the requirement of a power source introduces considerably more obstacles to the motion.…”

Character-Object Interaction Retrieval using the Interaction Bisector Surface

“…The interaction data used in our experiments includes both the motion of the character and the object. We capture the motions by following the approach by Sandilands et al [SCK13], where a magnetic motion capture system is used to track both the movements of the body parts and the object. The main advantage of using a magnetic motion capture system is that it does not suffer from occlusion problems.…”

“…We put 14 sensors on the actor and 2 sensors on the object. We also used 21 motion clips captured by Sandilands et al [SCK13], which can be downloaded from their project website. In total, there are 185 interactions with 14 types of objects.…”