Learning a tool's homogeneous transformation by tactile-based interaction

Li, Qiang; Haschke, Robert; Ritter, Helge

doi:10.1109/humanoids.2016.7803309

Cited by 4 publications

(5 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides the above tactile-based work, there are also studies that used geometric constraints and a series of contact to estimate in-hand object poses [30] [31] [32] [33] [34] [35]. The methods were clever but required multiple or continuous robotic actuation.…”

Section: B Tactile-based Methodsmentioning

confidence: 99%

In-Hand Pose Estimation Using Hand-Mounted RGB Cameras and Visuotactile Sensors

et al. 2023

View full text Add to dashboard Cite

This paper proposes a method to estimate the 6D pose of an object grasped by a robot hand using RGB cameras mounted on the palm and visuotactile sensors installed at the fingertips. It can deal with objects made from a wide range of materials thanks to combining the two types of sensors. The method allows a robot to robot to perform in-hand pose estimation while holding the object, eliminating the need for preparatory actions or particular environmental backgrounds. The mechanism at the back of the method includes deep-learning-based background subtraction and denoising auto-encoder-based sensor fusion. With the poses estimated using the proposed method, a robot controller can rectify the grasping uncertainty and adjust the robot motion to move an object toward required goals with satisfying accuracy. We conduct various studies and analyses in the experimental section to understand the proposed method's advantages and disadvantages. The results demonstrate the benefits of the proposed combination and mechanism. They also provide essential knowledge to readers considering using a similar configuration for estimating object poses.

show abstract

Section: B Tactile-based Methodsmentioning

confidence: 99%

In-Hand Pose Estimation Using Hand-Mounted RGB Cameras and Visuotactile Sensors

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Similar to tactile exploration, tactile grasping is also an important method to extract object properties. Some researchers extracted the contact level information [81], [91] and object level information [89], [105], [110], [126] Tactile Exploration analytical models [66], [97] [123], [156], [157] [72], [138], [158] data-driven [75] [159] [160] [161] Grasping analytical models [81], [91] [89], [105], [110], [126] [162], [163] data-driven [164], [165] [166], [167] [134], [168]- [170] In-hand Manipulation analytical models [171] [172] [173], [174], [137], [175], [176], [177] data-driven [154] [178] [179], [151] Tool Manipulation analytical models [180] [181] data-driven [29] [139] [182] [183], [143], [153] Locomotion analytical models [184], [95], [185], [186] -…”

Section: B Graspingmentioning

confidence: 99%

“…A key aspect of employing tactile sensing in tool manipulation is that the task contacts are between an object and the held tool. The contacts are therefore not directly on the tactile sensors, unless the tool is itself instrumented [180].…”

Section: Tool Manipulationmentioning

confidence: 99%

A Review of Tactile Information: Perception and Action Through Touch

Kroemer

et al. 2020

IEEE Trans. Robot.

Self Cite

185

View full text Add to dashboard Cite

Tactile sensing is a key sensor modality for robots interacting with their surroundings. These sensors provide a rich and diverse set of data signals that contain detailed information collected from contacts between the robot and its environment. The data is however not limited to individual contacts and can be used to extract a wide range of information about the objects in the robots environment as well as the robots own actions during the interactions.In this paper, we provide an overview of tactile information and its applications in robotics. We present a hierarchy consisting of raw, contact, object, and action levels to structure the tactile information, with higher-level information often building upon lower-level information. We discuss different types of information that can be extracted at each level of the hierarchy. The paper also includes an overview of different types of robot applications and the types of tactile information that they employ.The paper concludes with a discussion of tactile-based computational framework and future tactile applications which are still beyond current robot's capabilities.

show abstract

“…Using this strategy, Stückler and Behnke [24] developed many visionbased applications for everyday tools. Li et al [13] estimated the coordinate frame of a tactile tool by tactile-interaction. Combining the estimated coordinate frame with the robot's kinematic model, the robot was able to use the tool for tactile servoing.…”

Section: State Of the Artmentioning

confidence: 99%

“…There are two main approaches to study how to integrate a tool with the robot (1) geometry-based method and (2) experience-based learning method. For geometry-based method, the task of the robot is to automatically estimate the kinematic parameters of the tool [10] [13]. Given predefined motion patterns to move the grasped tool, the robot will use its vision or other modality to observe the relation between its end-effector and the tool's coordinate frame to derive the tool's kinematic parameters.…”

Section: Introductionmentioning

confidence: 99%

Estimating an Articulated Tool's Kinematics via Visuo-Tactile Based Robotic Interactive Manipulation

Ückermann

Haschke

et al. 2018

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

The usage of articulated tools for autonomous robots is still a challenging task. One of the difficulties is to automatically estimate the tool's kinematics model. This model cannot be obtained from a single passive observation, because some information, such as a rotation axis (hinge), can only be detected when the tool is being used. Inspired by a baby using its hands while playing with an articulated toy, we employ a dual arm robotic setup and propose an interactive manipulation strategy based on visual-tactile servoing to estimate the tool's kinematics model. In our proposed method, one hand is holding the tool's handle stably, and the other arm equipped with tactile finger flips the movable part of the articulated tool. An innovative visuo-tactile servoing controller is introduced to implement the flipping task by integrating the vision and tactile feedback in a compact control loop. In order to deal with the temporary invisibility of the movable part in camera, a data fusion method which integrates the visual measurement of the movable part and the fingertip's motion trajectory is used to optimally estimate the orientation of the tool's movable part. The important tool's kinematic parameters are estimated by geometric calculations while the movable part is flipped by the finger. We evaluate our method by flipping a pivoting cleaning head (flap) of a wiper and estimating the wiper's kinematic parameters. We demonstrate that the flap of the wiper is flipped robustly, even the flap is shortly invisible. The orientation of the flap is tracked well compared to the ground truth data. The kinematic parameters of the wiper are estimated correctly.

show abstract

Learning a tool's homogeneous transformation by tactile-based interaction

Cited by 4 publications

References 15 publications

In-Hand Pose Estimation Using Hand-Mounted RGB Cameras and Visuotactile Sensors

In-Hand Pose Estimation Using Hand-Mounted RGB Cameras and Visuotactile Sensors

A Review of Tactile Information: Perception and Action Through Touch

Estimating an Articulated Tool's Kinematics via Visuo-Tactile Based Robotic Interactive Manipulation

Contact Info

Product

Resources

About