Combining monoSLAM with object recognition for scene augmentation using a wearable camera

Castle, Robert O.; Klein, Georg; Murray, David

doi:10.1016/j.imavis.2010.03.009

Cited by 69 publications

(42 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While most approaches use SLAM as a front-end to obtain a sensor trajectory estimate (Zender et al 2008;Vasudevan et al 2007;Meger et al 2008;Nüchter and Hertzberg 2008;Castle et al 2010;Civera et al 2011), some methods also incorporate the spatial relation of objects into SLAM. Tomono and Shin'ichi (2003), for example, detect polyhedral object models in images and perform SLAM in 2D maps using the detected objects as landmarks.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Dense real-time mapping of object-class semantics from RGB-D video

Stückler

Waldvogel

Schulz

et al. 2013

J Real-Time Image Proc

View full text Add to dashboard Cite

We propose a real-time approach to learning semantic maps from moving RGB-D cameras. Our method models geometry, appearance, and semantic labeling of surfaces. We recover camera pose using simultaneous localization and mapping while concurrently recognizing and segmenting object classes in the images. Our object-class segmentation approach is based on random decision forests and yields a dense probabilistic labeling of each image. We implemented it on GPU to achieve a high frame rate. The probabilistic segmentation is fused in octree-based 3D maps within a Bayesian framework. In this way, image segmentations from various view points are integrated within a 3D map which improves segmentation quality. We evaluate our system on a large benchmark dataset and demonstrate state-of-the-art recognition performance of our object-class segmentation and semantic mapping approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

“…In our approach, we segment the original image data and fuse segmentation evidence from multiple views. Castle et al (2010) and Civera et al (2011) propose vision-based mapping of objects. In both approaches, SLAM is solved through feature-based monocular EKF-SLAM.…”

Section: Related Workmentioning

confidence: 99%

Dense real-time mapping of object-class semantics from RGB-D video

Stückler

Waldvogel

Schulz

et al. 2013

J Real-Time Image Proc

View full text Add to dashboard Cite

show abstract

“…While the robot scans the environment using 3D cameras, subse− quent images should be aligned and merged to gain the detailed information about the surrounding objects. Most of the recent SLAM techniques use a simple RGB camera [14][15][16], a laser measurement system [17,18] or 3D laser data [19]. The description of the 3D scans registration tech− niques can be found in the work of Andreasson [20].…”

Section: Related Workmentioning

confidence: 99%

On-line range images registration with GPGPU

Bedkowski

Naruniec

2013

Opto-Electronics Review

View full text Add to dashboard Cite

This paper concerns implementation of algorithms in the two important aspects of modern 3D data processing: data registration and segmentation. Solution proposed for the first topic is based on the 3D space decomposition, while the latter on image processing and local neighbourhood search. Data processing is implemented by using NVIDIA compute unified device architecture (NIVIDIA CUDA) parallel computation. The result of the segmentation is a coloured map where different colours correspond to different objects, such as walls, floor and stairs. The research is related to the problem of collecting 3D data with a RGB-D camera mounted on a rotated head, to be used in mobile robot applications. Performance of the data registration algorithm is aimed for on-line processing. The iterative closest point (ICP) approach is chosen as a registration method. Computations are based on the parallel fast nearest neighbour search. This procedure decomposes 3D space into cubic buckets and, therefore, the time of the matching is deterministic. First technique of the data segmentation uses accele-rometers integrated with a RGB-D sensor to obtain rotation compensation and image processing method for defining pre-requisites of the known categories. The second technique uses the adapted nearest neighbour search procedure for obtaining normal vectors for each range point.

show abstract

“…The closest approach to this paper is the work by Castle et al [3], [4], which registers planar objects into an EKF-based monocular SLAM. Their research make use of SIFT features to construct the appearance model of the objects and insert in the SLAM map three of the boundary corners of the plane.…”

Section: Related Workmentioning

confidence: 99%

“…In figure 6.d the bed has also been recognized and registered. Finally, figure 6.e shows the final frame of the sequence along with the final estimation results 3 . For a better visualization, figure 7 shows a detail of the final map estimation and the camera trajectory.…”

Section: B Hospital Room Environment -Roboearth Projectmentioning

confidence: 99%

Towards semantic SLAM using a monocular camera

Civera

Gálvez-López

Riazuelo

et al. 2011

2011 IEEE/RSJ International Conference on Intelligent Robots and Systems

154

View full text Add to dashboard Cite

Abstract-Monocular SLAM systems have been mainly focused on producing geometric maps just composed of points or edges; but without any associated meaning or semantic content. In this paper, we propose a semantic SLAM algorithm that merges in the estimated map traditional meaningless points with known objects. The non-annotated map is built using only the information extracted from a monocular image sequence. The known object models are automatically computed from a sparse set of images gathered by cameras that may be different from the SLAM camera. The models include both visual appearance and tridimensional information. The semantic or annotated part of the map -the objects-are estimated using the information in the image sequence and the precomputed object models.The proposed algorithm runs an EKF monocular SLAM parallel to an object recognition thread. This latest one informs of the presence of an object in the sequence by searching for SURF correspondences and checking afterwards their geometric compatibility. When an object is recognized it is inserted in the SLAM map, being its position measured and hence refined by the SLAM algorithm in subsequent frames.Experimental results show real-time performance for a handheld camera imaging a desktop environment and for a camera mounted in a robot moving in a room-sized scenario.

show abstract

Combining monoSLAM with object recognition for scene augmentation using a wearable camera

Cited by 69 publications

References 37 publications

Dense real-time mapping of object-class semantics from RGB-D video

Dense real-time mapping of object-class semantics from RGB-D video

On-line range images registration with GPGPU

Towards semantic SLAM using a monocular camera

Contact Info

Product

Resources

About