Vision and force/torque integration for realtime estimation of fast-moving object under intermittent contacts

Abstract

This paper considers the fast and accurate estimation of motion variables of a rigid body object whose movement occurs from intermittent contacts with coordinating manipulators in nonprehensile manipulation tasks. The estimator operates under multiple sensory data including visual, joint torque, joint position and/or tactile measurements which are combined at the lower level to compensate for the latency and the slow sampling of the visual data. The estimator is real-time in the sense that it provides the motion data of the target object at the same fast sample rate of the servo controller without delay. The basic formulation is the multi-rate Kalman filter with the contact force vector as its process input, and the visual observation as its measurement output signal which is the down-sampled and delayed version of the configuration of the target object. Experimental tests are conducted for the case of planar object manipulation as well as the non-centroidal rotation under gravity using a robotic hand, and their results are presented to demonstrate the validity of the proposed estimation scheme.

Nonprehensile Manipulation for Rapid Object Spinning via Multisensory Learning from Demonstration

“…Other types of sensory data adopted in recent years include tactile and vision data [ 26 , 27 ]. Closing a feedback loop with such sensors may enable the control law to adapt to unknown object properties [ 28 ]. In particular, tactile sensory information can play an important role in the process of manipulation by detecting both the direction and the magnitude of the contact force between the object and the robotic fingers.…”

“…Other types of sensory data adopted in recent years include tactile and vision data [26,27]. Closing a feedback loop with such sensors may enable the control law to adapt to unknown object properties [28].…”