Pointing It Out! Comparing Manual Segmentation of 3D Point Clouds between Desktop, Tablet, and Virtual Reality

Keep the Human in the Loop: Arguments for Human Assistance in the Synthesis of Simulation Data for Robot Training

“…Although mapping scan data to semantically meaningful objects is often performed using 2D interfaces, VR was found to be promising due to its 3D environment, which makes navigation easier [8]. Using VR, prior works proposed various methods for object labeling using boundary boxes [6], point cloud part selection and labeling throughout recorded video streams [7], or a tablet to enhance the selection accuracy for fine-grain adjustments [23].…”

“…Such intervention could also be performed iteratively with databases or generative algorithms (as in VRFromX [26]) to harness the respective strengths of humans and computers. Application experts could further contribute to the segmentation of objects from environmental scans such as point clouds by manually selecting objects, adjusting mesh boundaries, colors, and labels, or removing erroneous data [8,46] if a computed segmentation is incorrect or cannot be performed. The next stage involves creating meshes from the identified objects to enable interaction with virtual objects.…”

“…There are multiple technologies available through which application experts can be included in the robot training process, such as desktop solutions and MR technologies [25]. However, MR technologies, such as Augmented Reality (AR) and Virtual Reality (VR), offer great potential because they facilitate intuitive interaction with visual representations in three-dimensional space [8,29]. Thus, we choose to use VR to enable remote operation, allowing persons not physically present to interact with the system.…”

“…In recent years, human-centered approaches have revealed ways to utilize human perception and domain knowledge to enhance related approaches and artifacts. For instance, humans have assisted with object labeling [6], point cloud part selection, and labeling [7,8] and interactive segmentation using touch and voice commands [9] in Virtual Reality (VR). Thus, regarding robot training, we envision that end users can easily assist in specifying object existence, spatial movability, or material properties to enhance current approaches to generating realistic simulation environments.…”

The Interaction Efficiency of Different Visual Areas on a Virtual Reality Device Screen: Standing versus Sitting Posture

“…In recent years, VR devices have been progressively incorporated into everyday life, and their potential in elds such as healthcare, education, and rehabilitation is increasingly being explored (Cipresso et al, 2018;Fu & Li, 2023;Tabbaa et al, 2021). They can present more complex threedimensional information from multiple perspectives than traditional display devices, thereby offering a more immersive experience (Liebers et al, 2023). The intricate display formations also require users to adopt changing postures when interacting with VR devices.…”