A Work Area Visualization by Multi-View Camera-Based Diminished Reality

Abstract

Hand-held tools are indispensable for efficient manual working in fields ranging from woodworking to surgery. In this paper, we present a diminished reality (DR) method to visualize areas occluded by hands and tools in various hand-working scenarios. We propose a redesigned existing arbitrary viewpoint image generation method for DR applications as a core DR background rendering to recover views without undesirable objects. We conducted quantitative and qualitative experiments using real data to validate the performance of the proposed method. The experimental results showed that our method runs in real-time (40.1 fps) and surpasses conventional methods (including image inpainting-based and geometry-based approaches) in terms of image similarity measures.

Augmented Reality Views for Occluded Interaction

“…Such dynamic cameras enable users to control the camera's orientation and location, allowing the users to select a perspective relevant to the task at hand. Some techniques from augmented reality allow for arbitrary viewpoint positioning (e.g., [16,22]). Being able to control the remote camera's perspective can alleviate the frame-of-reference issues, while at the same time causing a new set of problems related to viewpoint instability and counter-intuitive control of the remote view.…”

An overview of augmented visualization: observing the real world as desired

“…We can simply select and warp one of these images onto the video stream from a cinematography camera to hide the newly placed visually annoying object. an overview of augmented visualization: observing the real world as desired [46] Limited field of view Wide range RGB-D sensor Polygon mesh rendering Smartphones/HUDs Work area visualization [47] Dynamic scene visualization Arbitrary arranged cameras Detour LFR HMDs De-fencing [48] Fence region restoration Sweep motion video Fence-aware LFR Smartphones (a) (b) However, when the background image is synthesized to overlay the object, photometric inconsistency between the synthesized and the surrounding regions occurs. Since the illumination conditions are different from each other, the simple overlay generates conspicuous borders around the ROI, as shown in Fig.…”

“…2) Example solution Figure 4 shows an example of visualizing a work area occluded by an operator's hand and a holding tool in the perspective [47,53]. In this setting, we had no space to place cameras below the hand or the tool, and therefore, we surrounded the workspace to capture the whole area.…”

InvisibleRobot: Facilitating Robot Manipulation Through Diminished Reality

“…In particular, we apply diminished reality (DR) to remove the occluding robot from the user's view and allow direct manipulation of the object. DR has previously been considered in a variety of tasks, such as X-Ray vision [5], manufacturing [4], and teleoperation [6]. However, these systems were designed for video see-through systems and are not viable for in-situ robot control due to physical constraints of mobile devices and safety concerns of video see-through head-mounted displays.…”