Robust Texture Mapping Using RGB-D Cameras

Oliveira, Mauro; Lim, Gi Hyun; Madeira, Tiago; Dias, Paulo; Santos, Vítor

doi:10.3390/s21093248

Cited by 5 publications

(5 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Currently, we use a per-pixel screen space texturing method for the final mesh to demonstrate our system (Section 3.5), which is prone to errors with slight misalignments of the calibration and requires high-quality depth images. The integration of existing work for consistent texturing [77][78][79] would be very beneficial to increase visual fidelity and consistency, which is vital for medical telepresence since many features and tools are too small to resolve with geometry using depth cameras. The enhancement of depth images [80][81][82][83] could also significantly increase the quality of our visual result and the time warping procedure presented in FusionMLS [15] to merge depth images at different recording timestamps better.…”

Section: Discussionmentioning

confidence: 99%

Real-Time 3D Reconstruction Pipeline for Room-Scale, Immersive, Medical Teleconsultation

Eck,

Wechner,

Pankratz

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

Medical teleconsultation was among the initial use cases for early telepresence research projects since medical treatment often requires timely intervention by highly specialized experts. When remote medical experts support interventions, a holistic view of the surgical site can increase situation awareness and improve team communication. A possible solution is the concept of immersive telepresence, where remote users virtually join the operating theater that is transmitted based on a real-time reconstruction of the local site. Enabled by the availability of RGB-D sensors and sufficient computing capability, it becomes possible to capture such a site in real time using multiple stationary sensors. The 3D reconstruction and simplification of textured surface meshes from the point clouds of a dynamic scene in real time is challenging and becomes infeasible for increasing capture volumes. This work presents a tightly integrated, stateless 3D reconstruction pipeline for dynamic, room-scale environments that generates simplified surface meshes from multiple RGB-D sensors in real time. Our algorithm operates directly on the fused, voxelized point cloud instead of populating signed-distance volumes per frame and using a marching cube variant for surface reconstruction. We extend the formulation of the dual contouring algorithm to work for point cloud data stored in an octree and interleave a vertex-clustering-based simplification before extracting the surface geometry. Our 3D reconstruction pipeline can perform a live reconstruction of six incoming depth videos at their native frame rate of 30 frames per second, enabling the reconstruction of smooth movement. Arbitrarily complex scene changes are possible since we do not store persistent information between frames. In terms of mesh quality and hole filling, our method falls between the direct mesh reconstruction and expensive global fitting of implicit functions.

show abstract

Section: Discussionmentioning

confidence: 99%

Real-Time 3D Reconstruction Pipeline for Room-Scale, Immersive, Medical Teleconsultation

Eck,

Wechner,

Pankratz

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…The usage of pairwise mappings with occluded faces introduces incorrect photometrical data into the system, which undermines the color correction procedure. To tackle this problem, we use two filtering methods, as in [18]: z-buffering and depth consistency. Although these filtering methods aim to tackle the same problem, their scope of work is different, as will be explained below.…”

Section: Filtering Of Pairwise Mappingsmentioning

confidence: 99%

“…Several authors have proposed methodologies to carry out the fusion, based on different forms of weighted average of the contributions of textures in the image space [16,17]. However, these approaches are highly sensitive to inaccuracies in camera pose estimation, as even slight misalignments may generate ghost and blurring artifacts in the textures, which are not visually appealing [18]. Still, selecting a single image to be used for texture mapping raises the problem of choosing the most adequate one from a discrete set of possibilities and additionally, how to produce a consistent selection of images for all the faces in the 3D mesh.…”

Section: Introductionmentioning

confidence: 99%

A Robust 3D-Based Color Correction Approach for Texture Mapping Applications

Coelho

Dal’Col

Madeira

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

Texture mapping of 3D models using multiple images often results in textured meshes with unappealing visual artifacts known as texture seams. These artifacts can be more or less visible, depending on the color similarity between the used images. The main goal of this work is to produce textured meshes free of texture seams through a process of color correcting all images of the scene. To accomplish this goal, we propose two contributions to the state-of-the-art of color correction: a pairwise-based methodology, capable of color correcting multiple images from the same scene; the application of 3D information from the scene, namely meshes and point clouds, to build a filtering procedure, in order to produce a more reliable spatial registration between images, thereby increasing the robustness of the color correction procedure. We also present a texture mapping pipeline that receives uncorrected images, an untextured mesh, and point clouds as inputs, producing a final textured mesh and color corrected images as output. Results include a comparison with four other color correction approaches. These show that the proposed approach outperforms all others, both in qualitative and quantitative metrics. The proposed approach enhances the visual quality of textured meshes by eliminating most of the texture seams.

show abstract

“…Drones play crucial roles in precision agriculture, including crop scouting for pest and disease identification, field mapping and planning, yield estimation, and irrigation management. Their diverse sensor technologies, ranging from RGB cameras to LiDAR sensors, allow for detailed data collection (Oliveira et al, 2021). Despite their benefits, challenges such as regulatory considerations, data processing complexities, and limited battery life persist.…”

Section: Drones (Unmanned Aerial Vehicles)mentioning

confidence: 99%

Advancements In Precision Agriculture Technologies For Enhancing Crop Yields

Ballabh,

Bhatt,

Joshi

et al. 2022

jsfs

View full text Add to dashboard Cite

Precision agriculture, a contemporary farming method, harnesses advanced technologies to optimize resource utilization, elevate productivity, and diminish environmental impact. The incorporation of diverse technologies, encompassing hardware, software, and data analytics solutions, is pivotal in accurate decision-making across all agricultural phases. Key technologies such as satellite imagery, drones, sensors, and IoT devices play integral roles in this process. these technologies, enabling farmers to tailor interventions at a granular level and optimize resources like water, fertilizers, and pesticides to enhance yields, reduce costs, and minimize environmental impact. However, persistent challenges including data interoperability, accessibility, and adoption restrictions underscore the need for ongoing research and cooperation to fully unlock the potential of precision agriculture in addressing global food security and sustainability challenges. Progress in remote sensing, including satellite imaging and drones, is explored, alongside the integration of GPS and GIS technology for precise field mapping and monitoring. The present review explores precision agriculture technologies, including remote sensing with satellite imaging, drones, GPS, GIS, and sensor applications for real-time data. It also emphasizes transformative potential in reshaping farming practices, enhancing resource efficiency, and contributing to sustainability.

show abstract

Robust Texture Mapping Using RGB-D Cameras

Cited by 5 publications

References 61 publications

Real-Time 3D Reconstruction Pipeline for Room-Scale, Immersive, Medical Teleconsultation

Real-Time 3D Reconstruction Pipeline for Room-Scale, Immersive, Medical Teleconsultation

A Robust 3D-Based Color Correction Approach for Texture Mapping Applications

Advancements In Precision Agriculture Technologies For Enhancing Crop Yields

Contact Info

Product

Resources

About