Image-guided ToF depth upsampling: a survey

Eichhardt, Ivan; Chetverikov, Dmitry; Jankó, Zsolt

doi:10.1007/s00138-017-0831-9

Cited by 31 publications

(30 citation statements)

References 131 publications

(206 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Spectacular SISR has been achieved using convolutional neural networks (CNNs) [22]. This success is mainly due to upscaling layers which are also used in semantic image segmentation [23] and 3D Time of Flight (ToF) upscaling [24]. These algorithms benefit greatly from the information contained in the scenes of a large set of images.…”

Section: Introductionmentioning

confidence: 99%

Hyperspectral demosaicking and crosstalk correction using deep learning

Dijkstra

Loosdrecht

Schomaker

et al. 2018

Machine Vision and Applications

View full text Add to dashboard Cite

Precision agriculture using unmanned aerial vehicles (UAVs) is gaining popularity. These UAVs provide a unique aerial perspective suitable for inspecting agricultural fields. With the use of hyperspectral cameras, complex inspection tasks are being automated. Payload constraints of UAVs require low weight and small hyperspectral cameras; however, such cameras with a multispectral color filter array suffer from crosstalk and a low spatial resolution. The research described in this paper aims to reduce crosstalk and to increase spatial resolution using convolutional neural networks. We propose a similarity maximization framework which is trained to perform end-to-end demosaicking and crosstalk-correction of a 4 × 4 raw mosaic image. The proposed method produces a hyperspectral image cube with 16 times the spatial resolution of the original cube while retaining a median structural similarity (SSIM) index of 0.85 (compared to an SSIM of 0.55 when using bilinear interpolation). Furthermore, this paper provides insight into the beneficial effects of crosstalk for hyperspectral demosaicking and gives best practices for several architectural and hyperparameter variations as well as a theoretical reasoning behind certain observations.

show abstract

Section: Introductionmentioning

confidence: 99%

Hyperspectral demosaicking and crosstalk correction using deep learning

Dijkstra

Loosdrecht

Schomaker

et al. 2018

Machine Vision and Applications

View full text Add to dashboard Cite

show abstract

“…One uses only single- or multi-frame low-res range images such as bicubic interpolation (Bicubic) proposed by Hou [ 6 ], convex projections proposed by Stark [ 7 ], neighbor embedding proposed by Chang [ 8 ] and self-guided residual interpolation proposed by Konno [ 9 ]. Another uses the information of high-res intensity images of the same scene to guide the SR of range images [ 10 ] such as joint bilateral filter (JBF) proposed by Kopf [ 11 ], weighted mode filtering proposed by Min [ 12 ], Markov random field (MRF) proposed by Diebel [ 13 ] and total generalized variation (TGV) proposed by Ferstl [ 14 ]. Considering that these algorithms must strictly align high-res intensity images and low-res range images, initially registering the two images through the registration algorithm is necessary.…”

Section: Introductionmentioning

confidence: 99%

Research on the Enhancement of Laser Radar Range Image Recognition Using a Super-Resolution Algorithm

Zhai

Jin

Xia

et al. 2020

Sensors

View full text Add to dashboard Cite

This work introduces a super-resolution (SR) algorithm for range images on the basis of self-guided joint filtering (SGJF), adding the range information of the range image as a coefficient of the filter to reduce the influence of the intensity image texture on the super-resolved image. A range image SR recognition system is constructed to study the effect of four SR algorithms including the SGJF algorithm on the recognition of the laser radar (ladar) range image. The effects of different model library sizes, SR algorithms, SR factors and noise conditions on the recognition are tested via experiments. Results demonstrate that all tested SR algorithms can improve the recognition rate of low-resolution (low-res) range images to varying degrees and the proposed SGJF algorithm has a very good comprehensive recognition performance. Finally, suggestions for the use of SR algorithms in actual scene recognition are proposed on the basis of the experimental results.

show abstract

“…Normal cameras, depth cameras [48][49][50], and panoramic cameras are commonly used optical imaging tools for the interior surface of a 3D object. The imaging field of view of the standard lens is limited, and the process of image capture is relatively complicated so as to cover the completed 3D object.…”

Section: Introductionmentioning

confidence: 99%

A 3D Reconstruction Pipeline of Urban Drainage Pipes Based on MultiviewImage Matching Using Low-Cost Panoramic Video Cameras

Zhang

Zhao

et al. 2019

Water

View full text Add to dashboard Cite

Urban drainage pipe networks have complex spatial contributions, andthey are now facing problems such as damage, defects, and aging. A rapid and high-precision pipe inspection strategy is thekey to ensuring thesustainable development of urban water supply and drainage system. In this paper, a three-dimensional (3D) reconstruction pipeline of urban drainage pipes based on multiview image matching using low-cost panoramic video cameras is proposed, which provides an innovative technical approach for pipe inspection. Firstly, we extracted frames from the panoramic video of the pipes andcorrected the geometric distortion using a spherical reprojection to obtain multiview pipe images. Second, the robust feature matching method using support lines and affine-invariant ratios isintroduced to conduct pipe image matching. Finally, the photogrammetric processing, using structure from motion (SfM) and dense reconstruction, wasintroduced to achieve the 3D modeling of drainage pipes. Several typical drainage pipes and shafts of the real scenes were taken for the 3D reconstruction experiments. Theresults show that our strategy can realize high-precision 3D reconstruction of different types of pipes, which can provide effective technical support for rapid and efficient inspection of urban pipes with broad application prospects in the daily management of sustainable urban drainage systems (SUDSs).

show abstract

Image-guided ToF depth upsampling: a survey

Cited by 31 publications

References 131 publications

Hyperspectral demosaicking and crosstalk correction using deep learning

Hyperspectral demosaicking and crosstalk correction using deep learning

Research on the Enhancement of Laser Radar Range Image Recognition Using a Super-Resolution Algorithm

A 3D Reconstruction Pipeline of Urban Drainage Pipes Based on MultiviewImage Matching Using Low-Cost Panoramic Video Cameras

Contact Info

Product

Resources

About