RGB-D Semantic Segmentation and Label-Oriented Voxelgrid Fusion for Accurate 3D Semantic Mapping

Shi, Wenjun; Xu, Jingwei; Zhu, Dongchen; Zhang, Guanghui; Wang, Xianshun; Li, Jiamao; Zhanga, Xiaolin

doi:10.1109/tcsvt.2021.3056726

Cited by 36 publications

(7 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The image data were detected by the YOLOv5 algorithm to obtain information on the ICV states. When considering that the point cloud has a large amount of data, the VoxelGrid [ 29 ] filtering algorithm was selected to reduce the data load. Then, the target-level perception data of the multi-sensor were transformed by perspective-n-point (PNP) and camera calibration to a global co-ordinate system.…”

Section: Real-time Trajectory Prediction Methods For Intelligent Conn...mentioning

confidence: 99%

Real-Time Trajectory Prediction Method for Intelligent Connected Vehicles in Urban Intersection Scenarios

Wang

Liu

et al. 2023

Sensors

View full text Add to dashboard Cite

Intelligent connected vehicles (ICVs) have played an important role in improving the intelligence degree of transportation systems, and improving the trajectory prediction capability of ICVs is beneficial for traffic efficiency and safety. In this paper, a real-time trajectory prediction method based on vehicle-to-everything (V2X) communication is proposed for ICVs to improve the accuracy of their trajectory prediction. Firstly, this paper applies a Gaussian mixture probability hypothesis density (GM-PHD) model to construct the multidimension dataset of ICV states. Secondly, this paper adopts vehicular microscopic data with more dimensions, which is output by GM-PHD as the input of LSTM to ensure the consistency of the prediction results. Then, the signal light factor and Q-Learning algorithm were applied to improve the LSTM model, adding features in the spatial dimension to complement the temporal features used in the LSTM. When compared with the previous models, more consideration was given to the dynamic spatial environment. Finally, an intersection at Fushi Road in Shijingshan District, Beijing, was selected as the field test scenario. The final experimental results show that the GM-PHD model achieved an average error of 0.1181 m, which is a 44.05% reduction compared to the LiDAR-based model. Meanwhile, the error of the proposed model can reach 0.501 m. When compared to the social LSTM model, the prediction error was reduced by 29.43% under the average displacement error (ADE) metric. The proposed method can provide data support and an effective theoretical basis for decision systems to improve traffic safety.

show abstract

Section: Real-time Trajectory Prediction Methods For Intelligent Conn...mentioning

confidence: 99%

Real-Time Trajectory Prediction Method for Intelligent Connected Vehicles in Urban Intersection Scenarios

Wang

Liu

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Lin et al [34] develop a multi-path refinement network (called RefineNet) to refine feature representations from high-level feature maps to low-level feature maps. Shi et al [5] leverage a two-stream network to fuse RGB and depth information for RGB-D semantic segmentation. Chen et al [35], [36] consider different importance levels of distinct classes and design an importance-aware loss for autonomous driving.…”

Section: A Accuracy-oriented Semantic Segmentation Methodsmentioning

confidence: 99%

“…Semantic image segmentation, which assigns a label from a set of predefined classes to each pixel in an image, is a fundamental technique to characterize the contextual relationship of semantic categories in street scenes [1]. It can be used as a pre-processing step to remove uninformative regions [2], [3] or combined with 3D scene geometry [4], [5]. In general, these tasks require not only highresolution input images to cover a wide field of view, but also fast inference speed for interaction or response.…”

Section: Acknowledgmentmentioning

confidence: 99%

Stage-Aware Feature Alignment Network for Real-Time Semantic Segmentation of Street Scenes

Weng

Yan

Chen

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Real-time performance is a very important trait of semantic segmentation models aiming at applications in robotics and intelligent transportation systems. Most previous work in the field involves custom convolutional encoders trained from scratch, and decoders without lateral skip-connections. However, we argue that a better speedaccuracy trade-off is achieved with i) compact encoders designed for competitive ImageNet performance and ii) lightweight decoders with lateral skip-connections. Additionally, we propose a novel interleaved pyramidal fusion scheme which is able to further improve the results on large objects close to the camera. We provide a detailed analysis of prediction accuracy and processing time on Cityscapes and CamVid datasets for models based on ResNet-18 and MobileNetv2. Our Cityscapes test submis-ages at 39.9 Hz on a GTX1080Ti. To the best of our knowledge, this result outperforms all previous approaches aiming at real-time application. The source code is available at https://github.com/orsic/swiftnet.

show abstract

“…As shown in Table 2, the Miao 11 algorithm has taken several different sets of original point cloud down-sampling parameter values for comparison. For the same ship point cloud model, the VoxelGrid 13 filter is used for down-sampling filtering with the grid size parameter value of 0.2, the result of down-sampling the original point cloud by twice the voxel is recorded as Miao 11 -1, the result after the original point cloud is down-sampled by the voxel is recorded as Miao 11 -2, and the result of the original point cloud without down-sampling is recorded as Miao 11 -3. It can be seen from Table 2 that after down-sampling the original point cloud data by 2 times the voxel as Miao 11 -1, the recognition accuracy reached the highest.…”

Section: Methodsmentioning

confidence: 99%

A fast recognition algorithm of ship hatch in bulk cargo terminal based on point cloud contour extraction

Yang

et al. 2022

Measurement and Control

View full text Add to dashboard Cite

Port loading automation systems can improve the efficiency of cargo transfer, save port operation time and create greater economic benefits. The recognition of ship hatch is the basis and premise of building an automatic loading system, it is a major time cost in the loading system meanwhile. How to identify the hatch quickly and accurately is an important problem that needs to be solved urgently under the actual production needs of ports. In order to save the time of ship hatch recognition, this paper proposes a fast hatch recognition algorithm based on point cloud contour extraction. The ship point cloud model generated by lidar scanning is preprocessed to remove the noise and isolated points in the model. Projecting the preprocessed point cloud on the XOY plane, converting the three-dimensional point cloud into a two-dimensional image, extracting the outline further getting the point cloud pixels with linear features of the two-dimensional image by [Formula: see text] algorithm. Projecting the feature point cloud to the X-axis to classify the hatches. According to the center point of each class of point cloud, searching for the nearest neighbor hatch edge feature points to calculate hatch coordinates in the world frame. An experimental study was carried out on the scan data of actual docked ships at Guoneng Tianjin Port. The results show that the algorithm can realize the hatch recognition quickly which the speed is increased by 424% compared with the previous algorithm and the identification accuracy meets the actual production needs of the port.

show abstract

RGB-D Semantic Segmentation and Label-Oriented Voxelgrid Fusion for Accurate 3D Semantic Mapping

Cited by 36 publications

References 52 publications

Real-Time Trajectory Prediction Method for Intelligent Connected Vehicles in Urban Intersection Scenarios

Real-Time Trajectory Prediction Method for Intelligent Connected Vehicles in Urban Intersection Scenarios

Stage-Aware Feature Alignment Network for Real-Time Semantic Segmentation of Street Scenes

A fast recognition algorithm of ship hatch in bulk cargo terminal based on point cloud contour extraction

Contact Info

Product

Resources

About