Implementation of Augmented Teleoperation System Based on Robot Operating System (ROS)

Lee, Donghyeon; Park, Young Soo

doi:10.1109/iros.2018.8594482

Cited by 24 publications

(22 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [103], AR was embedded to the control interface of a teleoperated aerial robot (Parrot Bebop quadcopter) programmed for data collection during environmental inspection. In [104], a novel teleoperator interface to remotely control the robot manipulators for nuclear waste cleanup is proposed. This interface could enhance the performance of a human operator using multi-model AR equipped with haptic feedback.…”

Section: Ar For Human-robot Interactionmentioning

confidence: 99%

Augmented Reality for Robotics: A Review

2020

View full text Add to dashboard Cite

Augmented reality (AR) is used to enhance the perception of the real world by integrating virtual objects to an image sequence acquired from various camera technologies. Numerous AR applications in robotics have been developed in recent years. The aim of this paper is to provide an overview of AR research in robotics during the five year period from 2015 to 2019. We classified these works in terms of application areas into four categories: (1) Medical robotics: Robot-Assisted surgery (RAS), prosthetics, rehabilitation, and training systems; (2) Motion planning and control: trajectory generation, robot programming, simulation, and manipulation; (3) Human-robot interaction (HRI): teleoperation, collaborative interfaces, wearable robots, haptic interfaces, brain-computer interfaces (BCIs), and gaming; (4) Multi-agent systems: use of visual feedback to remotely control drones, robot swarms, and robots with shared workspace. Recent developments in AR technology are discussed followed by the challenges met in AR due to issues of camera localization, environment mapping, and registration. We explore AR applications in terms of how AR was integrated and which improvements it introduced to corresponding fields of robotics. In addition, we summarize the major limitations of the presented applications in each category. Finally, we conclude our review with future directions of AR research in robotics. The survey covers over 100 research works published over the last five years.

show abstract

Section: Ar For Human-robot Interactionmentioning

confidence: 99%

Augmented Reality for Robotics: A Review

2020

View full text Add to dashboard Cite

show abstract

“…Augmented remote operation, or Augmented Telepresence (AT) [14], denotes applications where video-mediated communication is the enabling technology, but where additional data can be superimposed on or merged with the captured camera view as in AR. Such augmentation can be achieved on-site via using a see-through display and rendering only 3-Dimensional (3D) approximations of specific scene elements [15], or off-site via partial and full view rendering [16,17,9] for non-transparent displays. Unlike computer-generated imagery, this view rendering is at least partly based on the content of camera views.…”

Section: Related Workmentioning

confidence: 99%

“…The model is partly generated from the camera views, turning it into an extreme kind of view augmentation. This concept is taken further in [17], where the generated and presented 3D model of the scene is enhanced with virtual tracking markers overlaid in the presented 3D view.…”

Section: A Augmented Remote Operation In Non-entertainment Contextsmentioning

confidence: 99%

Camera and Lidar-Based View Generation for Augmented Remote Operation in Mining Applications

Dima

Sjöström

2021

IEEE Access

View full text Add to dashboard Cite

Remote operation of diggers, scalers, and other tunnel-boring machines has significant benefits for worker safety in underground mining. Real-time augmentation of the presented remote views can further improve the operator effectiveness through a more complete presentation of relevant sections of the remote location. In safety-critical applications, such augmentation cannot depend on preconditioned data, nor generate plausible-looking yet inaccurate sections of the view. In this paper, we present a capture and rendering pipeline for real time view augmentation and novel view synthesis that depends only on the inbound data from lidar and camera sensors. We suggest an on-the-fly lidar filtering for reducing point oscillation at no performance cost, and a full rendering process based on lidar depth upscaling and in-view occluder removal from the presented scene. Performance assessments show that the proposed solution is feasible for real-time applications, where per-frame processing fits within the constraints set by the inbound sensor data and within framerate tolerances for enabling effective remote operation.

show abstract

“…Even in these much-researched areas, there are still innovations, as the current works of (Gancet et al, 2015 ; Attard et al, 2018 ; Skilton et al, 2018 ; Klamt et al, 2019 ; Pervez et al, 2019 ) show. In addition, advances in telemanipulation are made possible by new technologies such as tactile feedback (Kuchenbecker et al, 2010 ; Fishel et al, 2016 ), augmented (Lee and Park, 2018 ), and virtual reality (Sagardia et al, 2015 ).…”

Section: Related Workmentioning

confidence: 99%

Kinesthetic Device vs. Keyboard/Mouse: A Comparison in Home Care Telemanipulation

et al. 2020

View full text Add to dashboard Cite

Ensuring care is one of the biggest humanitarian challenges of the future since an acute shortage in nursing staff is expected. At the same time, this offers the opportunity for new technologies in nursing, as the use of robotic systems. One potential use case is outpatient care, which nowadays involves traveling long distances. Here, the use of telerobotics could provide a major relief for the nursing staff, as it could spare them many of those—partially far—journeys. Since autonomous robotic systems are not desired at least in Germany for ethical reasons, this paper evaluates the design of a telemanipulation system consisting of off-the-shelf components for outpatient care. Furthermore, we investigated the suitability of two different input devices for control, a kinesthetic device, and a keyboard plus mouse. We conducted the investigations in a laboratory study. This laboratory represents a realistic environment of an elderly home and a remote care service center. It was carried out with 25 nurses. Tasks common in outpatient care, such as handing out things (manipulation) and examining body parts (set camera view), were used in the study. After a short training period, all nurses were able to control a manipulator with the two input devices and perform the two tasks. It was shown that the Falcon leads to shorter execution times (on average 0:54.82 min, compared to 01:10.92 min with keyboard and mouse), whereby the participants were more successful with the keyboard plus mouse, in terms of task completion. There is no difference in usability and cognitive load. Moreover, we pointed out, that the access to this kind of technology is desirable, which is why we identified further usage scenarios.

show abstract

Implementation of Augmented Teleoperation System Based on Robot Operating System (ROS)

Cited by 24 publications

References 17 publications

Augmented Reality for Robotics: A Review

Augmented Reality for Robotics: A Review

Camera and Lidar-Based View Generation for Augmented Remote Operation in Mining Applications

Kinesthetic Device vs. Keyboard/Mouse: A Comparison in Home Care Telemanipulation

Contact Info

Product

Resources

About