Label propagation in RGB-D video

Reza, Md. Alimoor; Zheng, Hui; Georgakis, Georgios V.; Košecká, Jana

doi:10.1109/iros.2017.8206371

Cited by 9 publications

(6 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [ 83 ], the authors proposed an approach for label propagation in RGB-D video sequences, in which each unlabelled frame is segmented using an intermediate 3D point cloud representation obtained from the camera pose and depth information of two keyframes. For similar purposes some studies deal with the 3D to 2D projection as can be seen for example in [ 84 ].…”

Section: Literature Reviewmentioning

confidence: 99%

Towards Semantic Photogrammetry: Generating Semantically Rich Point Clouds from Architectural Close-Range Photogrammetry

Murtiyoso

Pellis

Grussenmeyer

et al. 2022

Sensors

View full text Add to dashboard Cite

Developments in the field of artificial intelligence have made great strides in the field of automatic semantic segmentation, both in the 2D (image) and 3D spaces. Within the context of 3D recording technology it has also seen application in several areas, most notably in creating semantically rich point clouds which is usually performed manually. In this paper, we propose the introduction of deep learning-based semantic image segmentation into the photogrammetric 3D reconstruction and classification workflow. The main objective is to be able to introduce semantic classification at the beginning of the classical photogrammetric workflow in order to automatically create classified dense point clouds by the end of the said workflow. In this regard, automatic image masking depending on pre-determined classes were performed using a previously trained neural network. The image masks were then employed during dense image matching in order to constraint the process into the respective classes, thus automatically creating semantically classified point clouds as the final output. Results show that the developed method is promising, with automation of the whole process feasible from input (images) to output (labelled point clouds). Quantitative assessment gave good results for specific classes e.g., building facades and windows, with IoU scores of 0.79 and 0.77 respectively.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Towards Semantic Photogrammetry: Generating Semantically Rich Point Clouds from Architectural Close-Range Photogrammetry

Murtiyoso

Pellis

Grussenmeyer

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Our techniques are related to general work on semi-automatic video labeling. Most of these techniques start with manual annotations of a few keyframes, and then propagate those annotations across the remaining frames using cues such as spatial proximity, optical flow, or 3D reconstruction [10]- [12], [34], [35]. Many of these techniques are similar in nature to those used in object tracking.…”

Section: Other Related Workmentioning

confidence: 99%

“…A superpixel generation algorithm partitions the image into a reduced number of segments, thereby speeding up the work of subsequent processing which can process partitions instead of individual pixels. Reza et al [10] generated high-quality FIGURE 2. For each video frame, we identify candidate object masks using pre-trained object detectors (top branch).…”

Section: Superpixelsmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Dense Annotation for Monocular 3D Scene Understanding

et al. 2020

View full text Add to dashboard Cite

Deep neural networks have revolutionized many areas of computer vision, but they require notoriously large amounts of labeled training data. For tasks such as semantic segmentation and monocular 3d scene layout estimation, collecting high-quality training data is extremely laborious because dense, pixellevel ground truth is required and must be annotated by hand. In this paper, we present two techniques for significantly reducing the manual annotation effort involved in collecting large training datasets. The tools are designed to allow rapid annotation of entire videos collected by RGBD cameras, thus generating thousands of ground-truth frames to use for training. First, we propose a fully-automatic approach to produce dense pixel-level semantic segmentation maps. The technique uses noisy evidence from pre-trained object detectors and scene layout estimators and incorporates spatial and temporal context in a conditional random field formulation. Second, we propose a semi-automatic technique for dense annotation of 3d geometry, and in particular, the 3d poses of planes in indoor scenes. This technique requires a human to quickly annotate just a handful of keyframes per video, and then uses the camera poses and geometric reasoning to propagate these labels through an entire video sequence. Experimental results indicate that the technique could be used as an alternative or complementary source of training data, allowing large-scale data to be collected with minimal human effort.

show abstract

“…5). Budvytis et al [8] use semi-supervised learning to improve the intermediate labels and Reza et al [9] integrate depth and camera pose and formulate the problem as energy minimization in Conditional Random Fields.…”

Section: Introductionmentioning

confidence: 99%

Semantics through Time: Semi-supervised Segmentation of Aerial Videos with Iterative Label Propagation

Marcu¹,

Licăreţ²,

Costea³

et al. 2020

Preprint

View full text Add to dashboard Cite

Semantic segmentation is a crucial task for robot navigation and safety. However, current supervised methods require a large amount of pixelwise annotations to yield accurate results. Labeling is a tedious and time consuming process that has hampered progress in low altitude UAV applications. This paper makes an important step towards automatic annotation by introducing SegProp, a novel iterative flow-based method, with a direct connection to spectral clustering in space and time, to propagate the semantic labels to frames that lack human annotations. The labels are further used in semi-supervised learning scenarios. Motivated by the lack of a large video aerial dataset, we also introduce Ruralscapes, a new dataset with high resolution (4K) images and manually-annotated dense labels every 50 frames -the largest of its kind, to the best of our knowledge. Our novel SegProp automatically annotates the remaining unlabeled 98% of frames with an accuracy exceeding 90% (F-measure), significantly outperforming other state-of-the-art label propagation methods. Moreover, when integrating other methods as modules inside SegProp's iterative label propagation loop, we achieve a significant boost over the baseline labels. Finally, we test SegProp in a full semi-supervised setting: we train several state-of-the-art deep neural networks on the SegProp-automatically-labeled training frames and test them on completely novel videos. We convincingly demonstrate, every time, a significant improvement over the supervised scenario.

show abstract

Label propagation in RGB-D video

Cited by 9 publications

References 12 publications

Towards Semantic Photogrammetry: Generating Semantically Rich Point Clouds from Architectural Close-Range Photogrammetry

Towards Semantic Photogrammetry: Generating Semantically Rich Point Clouds from Architectural Close-Range Photogrammetry

Automatic Dense Annotation for Monocular 3D Scene Understanding

Semantics through Time: Semi-supervised Segmentation of Aerial Videos with Iterative Label Propagation

Contact Info

Product

Resources

About