Efficient Use of Textured 3D Model for Pre-observation-based Diminished Reality

A survey of diminished reality: Techniques for visually concealing, eliminating, and seeing through real objects

“…Various types of images are used as follows: Internet photos [13], X-ray images [8,24], image sets actively [25,26], streaming image sequences from surveillance cameras [27], multi-view cameras [6,[28][29][30], and RGB-D cameras [31,32]. One could store a sufficient number of viewpoints when one carefully captures a scene during enough time [13,25,26]. However, one will suffer from the geometric or photometric differences between the real and virtual scenes in the DR results due to the intervals between the preliminary observation and the current DR experience.…”

“…However, minimizing the ROI will reduce unnecessary artifacts while such rigorous edges may make the hidden area conspicuous. Geometric models of the target objects to be removed [25], simple 3D bounding boxes [6,26], and image recognition techniques [13,39] are used to detect the ROI. Several methods place the target objects in a non-reconstruction area not to be appeared in the resulting image [30,40].…”

“…In see-through, image-based methods are preferred for seamless real-virtual composition. Homography transformations of images [13,19,35,36,41,42], rendering texture 3D models [6,31,43], rendering voxels [37], and image-based rendering (IBR) [25,26,30,44] are the major approaches. In inpaint, pixels or image patches are synthesized in the ROI to fill in the region with plausible pixels [9,10,21,38].…”

“…Composition Make the real-virtual boundary less noticeable by post-processing or overlaying other effects on the DR image. Poisson blending-based methods [13,38] and pixel intensity estimation from surrounding pixels [26] are used for seamless real-virtual composition. Similarly, alpha blending at real-virtual boundaries is known to be effective [25,26].…”

“…Poisson blending-based methods [13,38] and pixel intensity estimation from surrounding pixels [26] are used for seamless real-virtual composition. Similarly, alpha blending at real-virtual boundaries is known to be effective [25,26]. For the replacing process, AR overlay is accomplished [20].…”