Reliable non-prehensile door opening through the combination of vision, tactile and force feedback

Abstract

Whereas vision and force feedback -either at the wrist or at the joint level -for robotic manipulation purposes has received considerable attention in the literature, the benefits that tactile sensors can provide when combined with vision and force have been rarely explored.In fact, there are some situations in which vision and force feedback cannot guarantee robust manipulation. Vision is frequently subject to calibration errors, occlusions and outliers, whereas force feedback can only provide useful information on those directions that are constrained by the environment. In tasks where the visual feedback contains errors, and the contact configuration does not constrain all the cartesian degrees of freedom, vision and force sensors are not sufficient to guarantee a successful execution.Many of the tasks performed in our daily life that do not require a firm grasp belong to this category. Therefore, it is important to develop strategies for robustly dealing with these situations. In this article, a new framework for combining tactile information with vision and force feedback is proposed and validated with the M. Prats Computer Science and Engineering Department Jaume-I University, Castellón, Spain E-mail: mprats@icc.uji.es P.J. Sanz Computer Science and Engineering Department Jaume-I University, Castellón, Spain E-mail: sanzp@icc.uji.es A.P. del Pobil Computer Science and Engineering Department Jaume-I University, Castellón, Spain and Department of Interaction Science Sungkyunkwan University, Seoul, South Korea E-mail: pobil@icc.uji.es task of opening a sliding door. Results show how the vision-tactile-force approach outperforms vision-force and force-alone, in the sense that it allows to correct the vision errors at the same time that a suitable contact configuration is guaranteed.

A unified multimodal control framework for human–robot interaction

“…hybrid force/position [22] or hybrid force/vision [20] control. This approach was recently extended in a framework integrating vision, force and tact to realize physical interaction tasks [32]. We hereby recall the formulation of that approach, and propose a more generic one, based on classic inverse kinematics control [28].…”

“…Let us now compare (5) with the hybrid sensors control used in numerous works [20,22,23,32]. This approach consists in assigning each sensing modality to a Cartesian direction in the operational space, and then summing the velocities associated with the selected sensors:…”

Model of tactile sensors using soft contacts and its application in robot grasping simulation

“…The relative velocity in the tangential direction of the contact plane can be obtained as follows:ṡ t =ṡ −ḋn P ij (6) Contact forces are described using the soft contact approach which allows small penetration between contacting bodies taking into account local deformations. The amount of this penetration is calculated accordingly with the maximum penetration defined for each sensor.…”

“…In reactive control, the robot has to cope with the inaccuracy of the vision systems when working in unstructured environments [4,5,6]. Recent studies have also shown how tactile-sensing-based algorithms can be employed to detect and react to contacts encountered during the execution of a grasp [7] as well as how tactile sensor information can be used to infer knowledge about grasp stability [8].…”

Novel Door-opening Method for Six-legged Robots Based on Only Force Sensing

“…Kobayashi, et al [20] applied an IP camera and IR distance sensors to calculate the position of the handle, which could be cylindrical with its diameter 48 mm to 56 mm or lever type. However, vision systems are frequently subject to calibration errors, occlusions and sight ranges, making it inevitable for scholars to apply force sensing to additionally double-confirm the contact position with the handle [21][22][23]. In fact, it is completely competent for robots to use only force sensing to detect the positional relationship with the door and the handle by touching at different positions and different directions, just like humans acting in the darkness.…”