Interactive Features for Robot Viewers

Hulin, Thomas; Hertkorn, Katharina; Preusche, Carsten

doi:10.1007/978-3-642-33503-7_19

Cited by 5 publications

(6 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To simplify and speed up the programming of such a complex robotic system, a separate interactive robot viewer [37] intuitively augments robot parameters over a virtual representation of the haptic device HUG (see Fig. 12).…”

Section: The Dlr Bimanual Haptic Device Hugmentioning

confidence: 99%

VR-OOS: The DLR's virtual reality simulator for telerobotic on-orbit servicing with haptic feedback

Sagardia

Hertkorn

Hulin

et al. 2015

2015 IEEE Aerospace Conference

Self Cite

View full text Add to dashboard Cite

The growth of space debris is becoming a severe issue that urgently requires mitigation measures based on maintenance, repair, and de-orbiting technologies. Such on-orbit servicing (OOS) missions, however, are delicate and expensive. Virtual Reality (VR) enables the simulation and training in a flexible and safe environment, and hence has the potential to drastically reduce costs and time, while increasing the success rate of future OOS missions. This paper presents a highly immersive VR system with which satellite maintenance procedures can be simulated interactively using visual and haptic feedback. The system can be used for verification and training purposes for human and robot systems interacting in space. Our framework combines unique realistic virtual reality simulation engines with advanced immersive interaction devices. The DLR bimanual haptic device HUG is used as the main user interface. The HUG is equipped with two light-weight robot arms and is able to provide realistic haptic feedback on both human arms. Additional devices provide vibrotactile and electrotactile feedback at the elbow and the fingertips. A particularity of the realtime simulation is the fusion of the Bullet physics engine with our haptic rendering algorithm, which is an enhanced version of the Voxmap-Pointshell Algorithm. Our haptic rendering engine supports multiple objects in the scene and is able to compute collisions for each of them within 1 msec, enabling realistic virtual manipulation tasks even for stiff collision configurations. The visualization engine ViSTA is used during the simulation to achieve photo-realistic effects, increasing the immersion. In order to provide a realistic experience at interactive frame rates, we developed a distributed system architecture, where the load of computing the physics simulation, haptic feedback and visualization of a complex scene is transferred to dedicated machines. The implementations are presented in detail and the performance of the overall system is validated. Additionally, a preliminary user study in which the virtual system is compared to a physical test bed shows the suitability of the VR-OOS framework.

show abstract

Section: The Dlr Bimanual Haptic Device Hugmentioning

confidence: 99%

VR-OOS: The DLR's virtual reality simulator for telerobotic on-orbit servicing with haptic feedback

Sagardia

Hertkorn

Hulin

et al. 2015

2015 IEEE Aerospace Conference

Self Cite

View full text Add to dashboard Cite

show abstract

“…In this work, two haptic devices, namely, a space qualified haptic device called the Space Joystick RJo (Artigas et al, 2016) and also a six Dof force feedback device, Lambda (Force Dimension), are integrated in order to teleoperate the LWR on SAM. This work's VR interface is based on Instant Player (Thomas et al, 2012), which is a lightweight software that runs on standard laptops without GPUs (enhancing portability). Instant Player also supports various hierarchies of a scene graph to create the required display.…”

Section: System Descriptionmentioning

confidence: 99%

“…A scene graph is general data structure with graph or tree like representations. It is used by the VR/AR softwares (Thomas et al, 2012), in order to produce the real-time 3D visualizations. Mathematically, let S be a scene graph.…”

Section: Object Pose Estimatormentioning

confidence: 99%

See 1 more Smart Citation

Virtual Reality via Object Pose Estimation and Active Learning: Realizing Telepresence Robots with Aerial Manipulation Capabilities

Lee¹,

Balachandran²,

Kondak³

et al. 2023

View full text Add to dashboard Cite

This paper presents a novel telepresence system for advancing aerial manipulation in dynamic and unstructured environments. The proposed system not only features a haptic device, but also a virtual reality (VR) interface that provides real-time 3D displays of the robot’s workspace as well as a haptic guidance to its remotely located operator. To realize this, multiple sensors, namely, a LiDAR, cameras, and IMUs are utilized. For processing of the acquired sensory data, pose estimation pipelines are devised for industrial objects of both known and unknown geometries. We further propose an active learning pipeline in order to increase the sample efficiency of a pipeline component that relies on a Deep Neural Network (DNN) based object detector. All these algorithms jointly address various challenges encountered during the execution of perception tasks in industrial scenarios. In the experiments, exhaustive ablation studies are provided to validate the proposed pipelines. Methodologically, these results commonly suggest how an awareness of the algorithms’ own failures and uncertainty (“introspection”) can be used to tackle the encountered problems. Moreover, outdoor experiments are conducted to evaluate the effectiveness of the overall system in enhancing aerial manipulation capabilities. In particular, with flight campaigns over days and nights, from spring to winter, and with different users and locations, we demonstrate over 70 robust executions of pick-andplace, force application and peg-in-hole tasks with the DLR cable-Suspended Aerial Manipulator (SAM). As a result, we show the viability of the proposed system in future industrial applications.

show abstract

“…The operator sees a combined 3D visualization of the remote and the virtual environment. The visualization uses interactive features for robot viewers [2] which provide important information about the remote robot (e.g. applied torques) intuitively, and presents results of the shared autonomy features.…”

mentioning

confidence: 99%