DigiTac: A DIGIT-TacTip Hybrid Tactile Sensor for Comparing Low-Cost High-Resolution Robot Touch

Lepora, Nathan F.; Lin, Yijiong; Ben, Money-Coomes,; Lloyd, John W.

doi:10.1109/lra.2022.3190641

Cited by 37 publications

(21 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Adversarial loss (10) where we set the hyperparameters α = 100, β = 200, and γ = 1, which are tuned experimentally. Finally, for training the adversarial discriminator D ψ , we use the cGAN objective as described in [36].…”

Section: E Real-to-simulation Generative Networkmentioning

confidence: 99%

“…In detail, the deformation of soft artificial skins upon physical contact with an object is detected through the optical tracking of visual features, such as markers or reflective membranes, which is then translated into tactile information, including contact location, force, vibration, object texture, and so on. The ViTac sensors have been found useful in small-scale manipulation tasks using robotics hands/fingers [9], [10]; however, their potential uses in large-scale whole-arm applications have not been comprehensively investigated.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Simulation, Learning, and Application of Vision-Based Tactile Sensing at Large Scale

Luu

Nguyen

2023

IEEE Trans. Robot.

View full text Add to dashboard Cite

Large-scale robotic skin with tactile sensing ability is emerging with the potential for use in close-contact human-robot systems. Although recent developments in vision-based tactile sensing and related learning methods are promising, they have been mostly designed for small-scale use, such as by fingers and hands, in manipulation tasks. Moreover, learning perception for such tactile devices demands a huge tactile dataset, which complicates the data collection process. To address this, this study introduces a multiphysics simulation pipeline, called SimTacLS, which considers not only the mechanical properties of external physical contact but also the realistic rendering of tactile images in a simulation environment. The system utilizes the obtained simulation dataset, including virtual images and skin deformation, to train a tactile deep neural network to extract high-level tactile information. Moreover, we adopt a generative network to minimize sim2real inaccuracy, preserving the simulation-based tactile sensing performance. Last but not least, we showcase this sim2real sensing method for our large-scale tactile sensor (TacLink) by demonstrating its use in two trial cases, namely, whole-arm nonprehensile manipulation and intuitive motion guidance, using a custom-built tactile robot arm integrated with TacLink. This article opens new possibilities in the learning of transferable tactile-driven robotics tasks from virtual worlds to actual scenarios without compromising accuracy.

show abstract

Section: E Real-to-simulation Generative Networkmentioning

confidence: 99%

mentioning

confidence: 99%

Simulation, Learning, and Application of Vision-Based Tactile Sensing at Large Scale

Luu

Nguyen

2023

IEEE Trans. Robot.

View full text Add to dashboard Cite

show abstract

“… Optical flow method [28], [37]  Finite element mode [63], [44]  Neural network [58], [53] mainly using the machine learningbased approaches  Speckle detection [77], [80]  Feature enhancement [81], [82] mainly using the physical model-based approaches  Stereo vision [25], [94]  Virtual stereo vision [23], [26] mainly using the physical model-based approaches  Contact area [24]  2D force distribution [17]  Slip field [79]  Contact area [20]  2D force distribution [86]  Slip field [21]  Contact area [91]  Friction coefficient [23]  2D force distribution [97] 3D tactile perception  Geometric features [46]  3D geometry [18]  3D force distribution [63]  Geometric features [82]  3D geometry [82]  3D force distribution [87]  Geometric features [ commonly used in the field of visuotactile sensing. By using a camera to photograph the marks prepared on the sensor contact elastomer, a tactile image containing the position change of the markers can be obtained, and the tactile information can be further obtained by post-processing and analyzing the tactile image.…”

Section: Common Technologiesmentioning

confidence: 99%

Marker Displacement Method Used in Vision-Based Tactile Sensors—From 2D to 3D: A Review

Li¹,

Li²,

Jiang³

2023

Preprint

View full text Add to dashboard Cite

<p>This article presents a detailed review and categorizing of the marker displacement method (MDM) used in vision-based tactile sensors. Vision-based tactile sensors have been proven to be a promising solution for robot tactile perception. Among such sensors, MDM is one of the most commonly used contact characterization and extraction methods. It uses visual approaches to obtain contact deformation and achieve multimodal tactile perception using physical models and post-processing algorithms. In recent years, many tactile sensors using MDM have been developed. However, the existing research does not strictly distinguish between the different types of methods but is uniformly grouped into MDM. Without differentiation, there might be a lack of systematic and comprehensive guidance in analyzing and optimizing the characteristics of MDM and selecting the most suitable method. This article is the first to classify MDM into three typical categories based on the dimensionality perspective: 2D MDM, 2.5D MDM, and 3D MDM. 2D MDM relies only on the monocular camera to acquire the marker array’s 2D displacement field. 2.5D MDM supplements 2D MDM with selected indirect features reflecting the location of the markers in the third dimension. 3D MDM employs a multi-camera system and can obtain the 3D displacement field using the stereo vision method common. Based on the latest literature, we compare the principles, characteristics, advantages and disadvantages, and applications of the three ways in detail. This work can provide a valuable reference for researchers interested in applying MDM in fields such as vision-based tactile sensors.</p>

show abstract

“…GelSight sensor [2] and various revised ones [4,8,9,11] use the photometric stereo technique [3] to measure highresolution 3D geometry of contact objects with a monocular camera. With a GelSight sensor mounted on the gripper, robots can accomplish challenging tasks such as texture recognition [12,13,14], dexterous manipulation [15,16,17,18], shape mapping [19,20], as well as liquid property estimation [21].…”

Section: Related Workmentioning

confidence: 99%

DTact: A Vision-Based Tactile Sensor that Measures High-Resolution 3D Geometry Directly from Darkness

Lin¹,

Lin²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Vision-based tactile sensors that can measure 3D geometry of the contacting objects are crucial for robots to perform dexterous manipulation tasks. However, the existing sensors are usually complicated to fabricate and delicate to extend. In this work, we novelly take advantage of the reflection property of semitransparent elastomer to design a robust, low-cost, and easy-to-fabricate tactile sensor named DTact. DTact measures high-resolution 3D geometry accurately from the darkness shown in the captured tactile images with only a single image for calibration. In contrast to previous sensors, DTact is robust under various illumination conditions. Then, we build prototypes of DTact that have non-planar contact surfaces with minimal extra efforts and costs. Finally, we perform two intelligent robotic tasks including pose estimation and object recognition using DTact, in which DTact shows large potential in applications.

show abstract

DigiTac: A DIGIT-TacTip Hybrid Tactile Sensor for Comparing Low-Cost High-Resolution Robot Touch

Cited by 37 publications

References 24 publications

Simulation, Learning, and Application of Vision-Based Tactile Sensing at Large Scale

Simulation, Learning, and Application of Vision-Based Tactile Sensing at Large Scale

Marker Displacement Method Used in Vision-Based Tactile Sensors—From 2D to 3D: A Review

DTact: A Vision-Based Tactile Sensor that Measures High-Resolution 3D Geometry Directly from Darkness

Contact Info

Product

Resources

About