Albedo estimation for real-time 3D reconstruction using RGB-D and IR data

Stotko, Patrick; Weinmann, Michael; Klein, Reinhard

doi:10.1016/j.isprsjprs.2019.01.018

Cited by 5 publications

(4 citation statements)

References 167 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A more direct option to handle the sparsity of appearance samples in reflectance estimation pipelines is the utilization of high-level scene understanding, e.g., in the form of a scene segmentation concerning objects or materials. The effectiveness of this technique has been demonstrated numerous times [Richter-Trummer et al, 2016;Azinovic et al, 2019;Bode et al, 2019;Stotko et al, 2019b] in recent years. While such a per-object segmentation is simple if the scene geometry is given as a mesh since topological information can be used to identify connected components, it is much more challenging in the case of popular point-based geometry representations using, e.g., disks [Whelan et al, 2016] or spheres [Lassner and Zollhofer, 2021] as underlying primitives.…”

Section: Edge and Boundary Detectionmentioning

confidence: 97%

“…Otherwise, such effects would be stored in the surface texture, which would lead to inconsistencies for scene relighting. To improve the quality of the reflectance reconstruction by separating the aforementioned effects in real-time, existing works exploit intrinsic image decomposition for (diffuse) albedo estimation [16,11,29,26,40]. These techniques achieve real-time capabilities at a reduced reconstruction accuracy.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Real-Time Multi-Material Reflectance Reconstruction for Large-Scale Scenes Under Uncontrolled Illumination from RGB-D Image Sequences

Bode

Merzbach

Stotko

et al. 2019

2019 International Conference on 3D Vision (3DV)

Self Cite

View full text Add to dashboard Cite

You made my studies a genuinely memorable experience. I am also very thankful for my internships at Meta Reality Labs Research in 2020 and 2021. Special thanks go to Michael Goesele, Yujia Chen, Christophe Hery, and Carlos Aliaga for giving me these amazing opportunities and supervising me during the internships.For proofreading this thesis, I want to thank Sebastian Kappes, Ralf Riesen, and Patrick Stotko.Finally, I thank all of my friends and family for their tremendous support throughout my studies. It was a big help to know that I can always rely on you.

show abstract

Section: Edge and Boundary Detectionmentioning

confidence: 97%

Section: Introductionmentioning

confidence: 99%

Real-Time Multi-Material Reflectance Reconstruction for Large-Scale Scenes Under Uncontrolled Illumination from RGB-D Image Sequences

Bode

Merzbach

Stotko

et al. 2019

2019 International Conference on 3D Vision (3DV)

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, the building cost of such systems is too high to make the acquisition process not available for casual users to have access to this acquisition process. To resolve this issue, practical methods using a single camera have been introduced [AWL15, RPG16, HSL*17, RRFG17, WZ15, WWZ16, SWK19, PNS18, NLGK18]. These methods can capture material appearance by inferring diffuse and specular appearance parameters from multiple observations with different view/light angles.…”

Section: Related Workmentioning

confidence: 99%

Progressive Acquisition of SVBRDF and Shape in Motion

Baek

Nam

et al. 2020

Computer Graphics Forum

View full text Add to dashboard Cite

To estimate appearance parameters, traditional SVBRDF acquisition methods require multiple input images to be captured with various angles of light and camera, followed by a post‐processing step. For this reason, subjects have been limited to static scenes, or a multiview system is required to capture dynamic objects. In this paper, we propose a simultaneous acquisition method of SVBRDF and shape allowing us to capture the material appearance of deformable objects in motion using a single RGBD camera. To do so, we progressively integrate photometric samples of surfaces in motion in a volumetric data structure with a deformation graph. Then, building upon recent advances of fusion‐based methods, we estimate SVBRDF parameters in motion. We make use of a conventional RGBD camera that consists of the colour and infrared cameras with active infrared illumination. The colour camera is used for capturing diffuse properties, and the infrared camera‐illumination module is employed for estimating specular properties by means of active illumination. Our joint optimization yields complete material appearance parameters. We demonstrate the effectiveness of our method with extensive evaluation on both synthetic and real data that include various deformable objects of specular and diffuse appearance.

show abstract

“…Indoor 3D environment perception technology is one of the key technologies for robot positioning and navigation, virtual reality, augmented reality and indoor mapping and localization [1][2][3][4][5][6][7]. With the rapid development of sensor technology, there are many devices that can be used for the point cloud acquisition and surface modeling of indoor scenes, such as LiDAR [8], RGB cameras [9], RGB-D cameras [5] and other commercial sensors, which are widely used in indoor 3D perception.…”

Section: Introductionmentioning

confidence: 99%

A Real-Time Infrared Stereo Matching Algorithm for RGB-D Cameras’ Indoor 3D Perception

Zhong

Liao

et al. 2020

IJGI

View full text Add to dashboard Cite

Low-cost, commercial RGB-D cameras have become one of the main sensors for indoor scene 3D perception and robot navigation and localization. In these studies, the Intel RealSense R200 sensor (R200) is popular among many researchers, but its integrated commercial stereo matching algorithm has a small detection range, short measurement distance and low depth map resolution, which severely restrict its usage scenarios and service life. For these problems, on the basis of the existing research, a novel infrared stereo matching algorithm that combines the idea of the semi-global method and sliding window is proposed in this paper. First, the R200 is calibrated. Then, through Gaussian filtering, the mutual information and correlation between the left and right stereo infrared images are enhanced. According to mutual information, the dynamic threshold selection in matching is realized, so the adaptability to different scenes is improved. Meanwhile, the robustness of the algorithm is improved by the Sobel operators in the cost calculation of the energy function. In addition, the accuracy and quality of disparity values are improved through a uniqueness test and sub-pixel interpolation. Finally, the BundleFusion algorithm is used to reconstruct indoor 3D surface models in different scenarios, which proved the effectiveness and superiority of the stereo matching algorithm proposed in this paper.

show abstract

Albedo estimation for real-time 3D reconstruction using RGB-D and IR data

Cited by 5 publications

References 167 publications

Real-Time Multi-Material Reflectance Reconstruction for Large-Scale Scenes Under Uncontrolled Illumination from RGB-D Image Sequences

Real-Time Multi-Material Reflectance Reconstruction for Large-Scale Scenes Under Uncontrolled Illumination from RGB-D Image Sequences

Progressive Acquisition of SVBRDF and Shape in Motion

A Real-Time Infrared Stereo Matching Algorithm for RGB-D Cameras’ Indoor 3D Perception

Contact Info

Product

Resources

About