Haptic Perception of Liquids Enclosed in Containers

Matl, Carolyn; Matthew, Robert Peter; Bajcsy, Růžena

doi:10.1109/iros40897.2019.8968528

Cited by 22 publications

(25 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Rendering. We employ the off-the-shelf rendering software: Pyrender [32] inside ArtiBoost. The interactive hand and object are the foreground, and the images in COCO [28] dataset are the background.…”

Section: Online Synthesis For Hope Taskmentioning

confidence: 99%

ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis

Li¹,

Yang²,

Zhan³

et al. 2021

Preprint

View full text Add to dashboard Cite

Estimating the articulated 3D hand-object pose from a single RGB image is a highly ambiguous and challenging problem requiring large-scale datasets that contain diverse hand poses, object poses, and camera viewpoints. Most real-world datasets lack this diversity. In contrast, synthetic datasets can easily ensure vast diversity, but learning from them is inefficient and suffers from heavy training consumption. To address the above issues, we propose ArtiBoost, a lightweight online data enrichment method that boosts articulated hand-object pose estimation from the data perspective. ArtiBoost is employed along with a real-world source dataset. During training, ArtiBoost alternatively performs data exploration and synthesis. Art-iBoost can cover various hand-object poses and camera viewpoints based on a Compositional hand-object Configuration and Viewpoint space (CCV-space) and can adaptively enrich the current hard-discernable samples by a mining strategy. We apply ArtiBoost on a simple learning baseline network and demonstrate the performance boost on several hand-object benchmarks. As an illustrative example, with ArtiBoost, even a simple baseline network can outperform the previous start-of-the-art based on Transformer on the HO3D dataset. Our code is available at https://github.com/MVIG-SJTU/ArtiBoost.

show abstract

Section: Online Synthesis For Hope Taskmentioning

confidence: 99%

ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis

Li¹,

Yang²,

Zhan³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Within the context of robotics, tactile sensing is widely recognized as important for dexterous manipulation [2] and a variety of robot tasks (e.g., grasping [34,6], slip-detection [9,44], handovers [16,17], and robot feeding [1]). However, prior work has focused on identifying contact events or properties of the grasped object (e.g., its texture [43], identity [1,39,38], or even its contents [7,29]). In this work, we focus on decoding vibratory signals that are transferred through the held/touched object.…”

Section: Background and Related Workmentioning

confidence: 99%

Extended Tactile Perception: Vibration Sensing through Tools and Grasped Objects

Taunyazov¹,

Shui²,

Lim³

et al. 2021

Preprint

View full text Add to dashboard Cite

Humans display the remarkable ability to sense the world through tools and other held objects. For example, we are able to pinpoint impact locations on a held rod and tell apart different textures using a rigid probe. In this work, we consider how we can enable robots to have a similar capacity, i.e., to embody tools and extend perception using standard grasped objects. We propose that vibro-tactile sensing using dynamic tactile sensors on the robot fingers, along with machine learning models, enables robots to decipher contact information that is transmitted as vibrations along rigid objects. This paper reports on extensive experiments using the BioTac micro-vibration sensor and a new event dynamic sensor, the NUSkin, capable of multitaxel sensing at 4 kHz. We demonstrate that fine localization on a held rod is possible using our approach (with errors less than 1 cm on a 20 cm rod). Next, we show that vibro-tactile perception can lead to reasonable grasp stability prediction during object handover, and accurate food identification using a standard fork. We find that multi-taxel vibro-tactile sensing at sufficiently high sampling rate (above 2 kHz) led to the best performance across the various tasks and objects. Taken together, our results provides both evidence and guidelines for using vibrotactile perception to extend tactile perception, which we believe will lead to enhanced competency with tools and better physical human-robot-interaction.

show abstract

“…In our simulator, all steps are performed in python across the robotics simulator PyBullet [12], the rendering tool Pyrender [36], and PyTorch [42]. For a given grasp and object, the object is loaded into PyBullet [12], along with a Wonik's Allegro Hand [48] equipped with vision-based touch sensors [33] on each of its fingers, and then the point in space corresponding to the action to be performed is selected and the grasping procedure is performed using PyBullet's physics simulator.…”

Section: Active Touch Simulatormentioning

confidence: 99%

“…A Pyrender [36] scene is kept constantly available throughout training to allow for touch signals to be simulated. The object is first loaded into the scene in the same position and orientation as in the PyBullet thread.…”

Section: A2 Simulating Touchmentioning

confidence: 99%

Active 3D Shape Reconstruction from Vision and Touch

Smith¹,

Meger²,

Villaseñor-Pineda³

et al. 2021

Preprint

View full text Add to dashboard Cite

Humans build 3D understandings of the world through active object exploration, using jointly their senses of vision and touch. However, in 3D shape reconstruction, most recent progress has relied on static datasets of limited sensory data such as RGB images, depth maps or haptic readings, leaving the active exploration of the shape largely unexplored. In active touch sensing for 3D reconstruction, the goal is to actively select the tactile readings that maximize the improvement in shape reconstruction accuracy. However, the development of deep learning-based active touch models is largely limited by the lack of frameworks for shape exploration. In this paper, we focus on this problem and introduce a system composed of: 1) a haptic simulator leveraging high spatial resolution vision-based tactile sensors for active touching of 3D objects; 2) a mesh-based 3D shape reconstruction model that relies on tactile or visuotactile signals; and 3) a set of data-driven solutions with either tactile or visuotactile priors to guide the shape exploration. Our framework enables the development of the first fully datadriven solutions to active touch on top of learned models for object understanding. Our experiments show the benefits of such solutions in the task of 3D shape understanding where our models consistently outperform natural baselines. We provide our framework as a tool to foster future research in this direction.

show abstract

Haptic Perception of Liquids Enclosed in Containers

Cited by 22 publications

References 24 publications

ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis

ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis

Extended Tactile Perception: Vibration Sensing through Tools and Grasped Objects

Active 3D Shape Reconstruction from Vision and Touch

Contact Info

Product

Resources

About