Integrating vision, haptics and proprioception into a feedback controller for in-hand manipulation of unknown objects

Li, Qiang; Elbrechter, Christof; Haschke, Robert; Ritter, Helge

doi:10.1109/iros.2013.6696703

Cited by 5 publications

(1 citation statement)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bimbo et al [17] combined vision and touch for the estimation of an object’s pose during in-hand manipulation where the camera suffers from finger occlusion. A visuotactile control was developed by Li et al [18], achieving robust in-hand manipulation and exploration of unknown objects performing robust manipulation even in the presence of accidental slippage or rolling. [19] presents another example of reliable grasps using underactuated hands, where flexible hands and visual clues were used to perform after grasping pose estimation for high-precision assembling.…”

Section: Literature Reviewmentioning

confidence: 99%

Estimating the Orientation of Objects from Tactile Sensing Data Using Machine Learning Methods and Visual Frames of Reference

Fonseca

Oliveira

Petriu

2019

Sensors

View full text Add to dashboard Cite

Underactuated hands are useful tools for robotic in-hand manipulation tasks due to their capability to seamlessly adapt to unknown objects. To enable robots using such hands to achieve and maintain stable grasping conditions even under external disturbances while keeping track of an in-hand object’s state requires learning object-tactile sensing data relationships. The human somatosensory system combines visual and tactile sensing information in their “What and Where” subsystem to achieve high levels of manipulation skills. The present paper proposes an approach for estimating the pose of in-hand objects combining tactile sensing data and visual frames of reference like the human “What and Where” subsystem. The system proposed here uses machine learning methods to estimate the orientation of in-hand objects from the data gathered by tactile sensors mounted on the phalanges of underactuated fingers. While tactile sensing provides local information about objects during in-hand manipulation, a vision system generates egocentric and allocentric frames of reference. A dual fuzzy logic controller was developed to achieve and sustain stable grasping conditions autonomously while forces were applied to in-hand objects to expose the system to different object configurations. Two sets of experiments were used to explore the system capabilities. On the first set, external forces changed the orientation of objects while the fuzzy controller kept objects in-hand for tactile and visual data collection for five machine learning estimators. Among these estimators, the ridge regressor achieved an average mean squared error of 0.077 ∘ . On the second set of experiments, one of the underactuated fingers performed open-loop object rotations and data recorded were supplied to the same set of estimators. In this scenario, the Multilayer perceptron (MLP) neural network achieved the lowest mean squared error of 0.067 ∘ .

show abstract

Section: Literature Reviewmentioning

confidence: 99%