PoseIt: A Visual-Tactile Dataset of Holding Poses for Grasp Stability Analysis

Kanitkar, Shubham; Jiang, Helen; Yuanl, Wenzhen

doi:10.1109/iros47612.2022.9981562

Cited by 7 publications

(4 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The PoseIt Dataset [143] provides a comprehensive collection of multi-modal sensor data for predicting the stability of a grasp in a holding pose. This dataset includes tactile, RGB, and force/torque sensor readings collected during a sequence of timesteps, with a constant gripping force across all timesteps.…”

Section: Datasets Availablementioning

confidence: 99%

Vision-Based Tactile Sensors in Precision Agriculture: Deep Learning Approaches, Applications, and Limitations

Fahmy,

Hassan,

Hussain

et al. 2024

Preprint

View full text Add to dashboard Cite

The integration of artificial intelligence with sensor technologies has revolutionized precision agriculture, offering unprecedented opportunities for enhancing crop management and productivity. This review focuses on the latest advancements in vision-based tactile sensors, a technology at the forefront of this transformation. By combining tactile data with vision-based techniques, these sensors provide a more comprehensive understanding of the agricultural environment. We investigate thoroughly the role of deep learning approaches in refining the functionality of these sensors, highlighting their potential to significantly improve the accuracy and efficiency of agricultural operations. The paper also explores the importance of specialized datasets in training deep neural networks for vision-based tactile applications, assessing the current landscape and identifying gaps in the available data. Through a thorough examination of the current state of the art, this review paper aims to shed light on the potential of AI-driven tactile sensing in precision agriculture and outline future research directions to further advance this field.

show abstract

Section: Datasets Availablementioning

confidence: 99%

Vision-Based Tactile Sensors in Precision Agriculture: Deep Learning Approaches, Applications, and Limitations

Fahmy,

Hassan,

Hussain

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Cui et al [5] utilized a 3D CNN-based visual-tactile fusion network to evaluate the grasp state of deformable objects. Kanitkar et al [7] presented a multimodal dataset consisting of visual-tactile information to investigate the impact of varied holding poses on grasp stability. Nevertheless, the adoption of simplistic feature-level fusion approaches in these studies resulted in a restricted exploitation of complementary information and an inability to effectively capture the interplay among unimodal features.…”

Section: A Grasp Stability Evaluationmentioning

confidence: 99%

“…𝐗 ℎ ← 𝐗 ℎ + MHSA(𝐗 ℎ + 𝐏 ℎ , 𝐗 𝑣 + 𝐏 𝑣 , 𝐗 𝑣 ) (7) where 𝐗 𝑣 and 𝐗 ℎ denote the feature sequences of the visual and tactile channels, respectively.…”

Section: Feature Extractionmentioning

confidence: 99%

“…They also introduced an MMFN that utilizes the self-attention mechanism [6]. In a recent study, Kanitkar et al [7] introduced a multimodal dataset that includes tactile and visual data to explore grasp outcomes at specific holding poses. However, the visual-tactile fusion-based grasp stability evaluation methods discussed above still exhibit certain limitations.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

ChinaSpec: a network for long-term in situ measurements of solar-induced fluorescence and reflectance in China

Zhang¹,

Zhang²,

Liu³

et al. 2020

Preprint

View full text Add to dashboard Cite

Extensive research has been conducted on assessing grasp stability, a crucial prerequisite for achieving optimal grasping strategies, including the minimum force grasping policy. However, existing works employ basic feature-level fusion techniques to combine visual and tactile modalities, resulting in the inadequate utilization of complementary information and the inability to model interactions between unimodal features. This work proposes an attention-guided cross-modality fusion architecture to comprehensively integrate visual and tactile features. This model mainly comprises convolutional neural networks (CNNs), self-attention, and cross-attention mechanisms. In addition, most existing methods collect datasets from real-world systems, which is time-consuming and high-cost, and the datasets collected are comparatively limited in size. This work establishes a robotic grasping system through physics simulation to collect a multimodal dataset. To address the sim-to-real transfer gap, we propose a migration strategy encompassing domain randomization and domain adaptation techniques. The experimental results demonstrate that the proposed fusion framework achieves markedly enhanced prediction performance (approximately 10%) compared to other baselines. Moreover, our findings suggest that the trained model can be reliably transferred to real robotic systems, indicating its potential to address real-world challenges.

show abstract

Grasp Stability Prediction with Time Series Data Based on STFT and LSTM

Wang,

Kirchner

2023

2023 International Conference on Advanced Robotics and Mechatronics (ICARM)

View full text Add to dashboard Cite

PoseIt: A Visual-Tactile Dataset of Holding Poses for Grasp Stability Analysis

Cited by 7 publications

References 30 publications

Vision-Based Tactile Sensors in Precision Agriculture: Deep Learning Approaches, Applications, and Limitations

Vision-Based Tactile Sensors in Precision Agriculture: Deep Learning Approaches, Applications, and Limitations

ChinaSpec: a network for long-term in situ measurements of solar-induced fluorescence and reflectance in China

Grasp Stability Prediction with Time Series Data Based on STFT and LSTM

Contact Info

Product

Resources

About