A Depth Camera-Based Soft Fingertip Device for Contact Region Estimation and Perception-Action Coupling

Huang, Isabella; Liu, Jingjun; Bajcsy, Růžena

doi:10.1109/icra.2019.8793612

Cited by 17 publications

(18 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [37], the change in diameter of the markers was used to measure the movement in the z direction. In [73,74], a depth camera is used to measure the shape of the soft fingertip, including the z direction. The marker is not embedded in the soft fingertip body, but the depth camera projects patterns on the inside of the sensor skin, and the principle is basically the same in this category.…”

Section: Physical Contact To Light Conversionmentioning

confidence: 99%

Tactile Image Sensors Employing Camera: A Review

Shimonomura

2019

Sensors

128

View full text Add to dashboard Cite

A tactile image sensor employing a camera is capable of obtaining rich tactile information through image sequences with high spatial resolution. There have been many studies on the tactile image sensors from more than 30 years ago, and, recently, they have been applied in the field of robotics. Tactile image sensors can be classified into three typical categories according to the method of conversion from physical contact to light signals: Light conductive plate-based, marker displacement- based, and reflective membrane-based sensors. Other important elements of the sensor, such as the optical system, image sensor, and post-image analysis algorithm, have been developed. In this work, the literature is surveyed, and an overview of tactile image sensors employing a camera is provided with a focus on the sensing principle, typical design, and variation in the sensor configuration.

show abstract

Section: Physical Contact To Light Conversionmentioning

confidence: 99%

Tactile Image Sensors Employing Camera: A Review

Shimonomura

2019

Sensors

128

View full text Add to dashboard Cite

show abstract

“…When the skin gets compressed after contact with an object, the scattering pattern of the light in the optical fibers changes, thus allowing for the measurement of force and the localization of the object. Huang et al [13] used a small depth sensor on robot's fingertip to directly image the structure of the soft contact region, as well as a rough reconstruction of the stress induced by geometric material strain.…”

Section: Vision-based Tactile Sensorsmentioning

confidence: 99%

ShadowSense

Bejarano

Hoffman

2020

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

View full text Add to dashboard Cite

This paper proposes and evaluates the use of image classification for detailed, full-body human-robot tactile interaction. A camera positioned below a translucent robot skin captures shadows generated from human touch and infers social gestures from the captured images. This approach enables rich tactile interaction with robots without the need for the sensor arrays used in traditional social robot tactile skins. It also supports the use of touch interaction with non-rigid robots, achieves high-resolution sensing for robots with different sizes and shape of surfaces, and removes the requirement of direct contact with the robot. We demonstrate the idea with an inflatable robot and a standing-alone testing device, an algorithm for recognizing touch gestures from shadows that uses Densely Connected Convolutional Networks, and an algorithm for tracking positions of touch and hovering shadows. Our experiments show that the system can distinguish between six touch gestures under three lighting conditions with 87.5 - 96.0% accuracy, depending on the lighting, and can accurately track touch positions as well as infer motion activities in realistic interaction conditions. Additional applications for this method include interactive screens on inflatable robots and privacy-maintaining robots for the home.

show abstract

“…Previously, research has focused on the optimization of sensor layout exploring how sensor structure can be optimized to maximize information gain ( Thuruthel et al, 2020 ), to allow the morphology to perform localized processing ( Hughes and Iida, 2017 ), to amplify or improve the sensitivity of the response ( Fend et al, 2004 ), and to optimized for a specific task ( Qi et al, 2019 ). In addition, the relationship between sensor morphology and action for perception has been explored ( Huang et al, 2019 ; Scimeca et al, 2020b ; Scimeca et al, 2021 ). These approaches highlight the importance of sensor structure and morphology when performing action based perception tasks ( Bernth et al, 2018 ).…”

Section: Introductionmentioning

confidence: 99%

Online Morphological Adaptation for Tactile Sensing Augmentation

et al. 2021

View full text Add to dashboard Cite

Sensor morphology and structure has the ability to significantly aid and improve tactile sensing capabilities, through mechanisms such as improved sensitivity or morphological computation. However, different tactile tasks require different morphologies posing a challenge as to how to best design sensors, and also how to enable sensor morphology to be varied. We introduce a jamming filter which, when placed over a tactile sensor, allows the filter to be shaped and molded online, thus varying the sensor structure. We demonstrate how this is beneficial for sensory tasks analyzing how the change in sensor structure varies the information that is gained using the sensor. Moreover, we show that appropriate morphology can significantly influence discrimination, and observe how the selection of an appropriate filter can increase the object classification accuracy when using standard classifiers by up to 28%.

show abstract

A Depth Camera-Based Soft Fingertip Device for Contact Region Estimation and Perception-Action Coupling

Cited by 17 publications

References 25 publications

Tactile Image Sensors Employing Camera: A Review

Tactile Image Sensors Employing Camera: A Review

ShadowSense

Online Morphological Adaptation for Tactile Sensing Augmentation

Contact Info

Product

Resources

About