CobotAR: Interaction with Robots using Omnidirectionally Projected Image and DNN-based Gesture Recognition

Nazarova, Elena; Oleg, Sautenkov,; Cabrera, Miguel Altamirano; Tirado, Jonathan; Serpiva, Valerii; Rakhmatulin, Viktor; Tsetserukou, Dzmitry

doi:10.1109/smc52423.2021.9658827

Cited by 5 publications

(2 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With the development of increasingly smaller and lighter projectors, new solutions have been proposed to overcome this problem. Nazarova et al, for instance, mounted a tracking and projection module on a collaborative robot to provide immersive and intuitive robot control to the user [ 13 ]. Hartmann’s team proposed an actuated Head-Mounted Projector (HMP) to increase the user’s field of view and foster collaboration and communication by providing a shared experience with other people [ 14 ].…”

Section: Introductionmentioning

confidence: 99%

Head-Mounted Projector for Manual Precision Tasks: Performance Assessment

Mamone

Ferrari

D’Amato

et al. 2023

Sensors

View full text Add to dashboard Cite

The growing interest in augmented reality applications has led to an in-depth look at the performance of head-mounted displays and their testing in numerous domains. Other devices for augmenting the real world with virtual information are presented less frequently and usually focus on the description of the device rather than on its performance analysis. This is the case of projected augmented reality, which, compared to head-worn AR displays, offers the advantages of being simultaneously accessible by multiple users whilst preserving user awareness of the environment and feeling of immersion. This work provides a general evaluation of a custom-made head-mounted projector for the aid of precision manual tasks through an experimental protocol designed for investigating spatial and temporal registration and their combination. The results of the tests show that the accuracy (0.6±0.1 mm of spatial registration error) and motion-to-photon latency (113±12 ms) make the proposed solution suitable for guiding precision tasks.

show abstract

Section: Introductionmentioning

confidence: 99%

Head-Mounted Projector for Manual Precision Tasks: Performance Assessment

Mamone

Ferrari

D’Amato

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Recent works on deploying Deep Neural Networks (DNN) for safe and secure automation (Biondi et al 2019;Nazarova et al 2021) propose a visionary hypervisor-centric architecture. Yet, the integration of ML modules is meticulously tailored for specific applications.…”

Section: Introductionmentioning

confidence: 99%

PerFC: An Efficient 2D and 3D Perception Software-Hardware Framework for Mobile Cobot

Dang

Nguyen

Huber

2023

FLAIRS

View full text Add to dashboard Cite

In this work, we present an end-to-end software-hardware framework that supports both conventional hardware and software components and integrates machine learning object detectors without requiring an additional dedicated graphic processor unit (GPU). We design our framework to achieve real-time performance on the robot system, guarantee such performance on multiple computing devices, and concentrate on code reusability. We then utilize transfer learning strategies for 2D object detection and fuse them into depth images for 3D depth estimation. Lastly, we test the proposed framework on the Baxter robot with two 7-DOF arms and a four-wheel mobility base. The results show that the robot achieves real-time performance while executing other tasks (map building, localization, navigation, object detection, arm moving, and grasping) with available hardware like Intel onboard GPUs on distributed computers. Also, to comprehensively control, program, and monitor the robot system, we design and introduce an end-user application. The source code is available at https://github.com/tuantdang/perception_framework.

show abstract

Gesture Control System for Desktop Robotic Arm

Lou

Wan

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

CobotAR: Interaction with Robots using Omnidirectionally Projected Image and DNN-based Gesture Recognition

Cited by 5 publications

References 10 publications

Head-Mounted Projector for Manual Precision Tasks: Performance Assessment

Head-Mounted Projector for Manual Precision Tasks: Performance Assessment

PerFC: An Efficient 2D and 3D Perception Software-Hardware Framework for Mobile Cobot

Gesture Control System for Desktop Robotic Arm

Contact Info

Product

Resources

About