RaLL: End-to-End Radar Localization on Lidar Map Using Differentiable Measurement Model

Yin, Huan; Chen, Runjian; Wang, Yue; Xiong, Rong

doi:10.1109/tits.2021.3061165

Cited by 23 publications

(45 citation statements)

References 56 publications

(50 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared to the cameras and laser scanners, radar sensor has already been used in the automotive industry (Krstanovic et al, 2012). With the development of Frequency-Modulated Continuous-Wave (FMCW) radar sensor 1 , the mapping and localization topics are studied in the recent years, for example the RadarSLAM (Hong et al, 2020), radar odometry (Cen and Newman, 2018;Barnes et al, 2020b), and radar localization on lidar maps (Yin et al, 2020(Yin et al, , 2021.…”

Section: Radar-based Mapping and Localizationmentioning

confidence: 99%

“…Our proposed lidar submap construction is able to reduce the differences of the two lidar equipments and settings on mobile robots. To demonstrate the generalization ability, we follow the training strategy of our previous work (Yin et al, 2021), in which only part of the RobotCar dataset is used for training. In the test stage, as shown in Figure 3, the learned model is evaluated on another part of the RobotCar and also generalized to MulRan-Riverside and MulRan-KAIST directly without retraining.…”

Section: Implementation and Experimental Setupmentioning

confidence: 99%

See 1 more Smart Citation

Radar-to-Lidar: Heterogeneous Place Recognition via Joint Learning

Yin

Wang

et al. 2021

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

Place recognition is critical for both offline mapping and online localization. However, current single-sensor based place recognition still remains challenging in adverse conditions. In this paper, a heterogeneous measurement based framework is proposed for long-term place recognition, which retrieves the query radar scans from the existing lidar (Light Detection and Ranging) maps. To achieve this, a deep neural network is built with joint training in the learning stage, and then in the testing stage, shared embeddings of radar and lidar are extracted for heterogeneous place recognition. To validate the effectiveness of the proposed method, we conducted tests and generalization experiments on the multi-session public datasets and compared them to other competitive methods. The experimental results indicate that our model is able to perform multiple place recognitions: lidar-to-lidar (L2L), radar-to-radar (R2R), and radar-to-lidar (R2L), while the learned model is trained only once. We also release the source code publicly: https://github.com/ZJUYH/radar-to-lidar-place-recognition.

show abstract

Section: Radar-based Mapping and Localizationmentioning

confidence: 99%

Section: Implementation and Experimental Setupmentioning

confidence: 99%

Radar-to-Lidar: Heterogeneous Place Recognition via Joint Learning

Yin

Wang

et al. 2021

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

show abstract

“…The authors demonstrated the effectiveness with synthetic tracking task and visual odometry in the real world. As for the range sensors, the differentiable Kalman filter can also be integrated to the end-to-end system for vehicle pose tracking [18]. Despite the Kalman filter above, some research works focused on building end-to-end particle filter [22,23].…”

Section: Related Workmentioning

confidence: 99%

“…Finally, we propose a Differentiable Kalman Filter to filter the state by optimally fusing NAE measurements and temporal dynamics, named NAE-DF. Thanks to the differentiability, the whole architecture can be trained in an end-to-end manner [17,18,19]. In this way, the uncertainty of the NAE measurement can also be indirectly supervised.…”

Section: Introductionmentioning

confidence: 99%

Neural Motion Prediction for In-flight Uneven Object Catching

Yu¹,

Guo²,

Yin³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

In-flight objects capture is extremely challenging. The robot is required to complete trajectory prediction, interception position calculation and motion planning in sequence within tens of milliseconds. As in-flight uneven objects are affected by various kinds of forces, motion prediction is difficult for a time-varying acceleration. In order to compensate the system's non-linearity, we introduce the Neural Acceleration Estimator (NAE) that estimates the varying acceleration by observing a small fragment of previous deflected trajectory. Moreover, end-to-end training with Differantiable Filter (NAE-DF) gives a supervision for measurement uncertainty and further improves the prediction accuracy. Experimental results show that motion prediction with NAE and NAE-DF is superior to other methods and has a good generalization performance on unseen objects. We test our methods on a robot, performing velocity control in real world and respectively achieve 83.3% and 86.7% success rate on a ploy urethane banana and a gourd. We also release an object in-flight dataset containing 1,500 trajectorys for uneven objects.

show abstract

“…Point cloud map [24], [25], [26], [27], [28], [29], [30], [31], [32] is a novel map source that provides dense and accurate 3D reference points. Compared with the OSM and satellite images, a point cloud map well supports the 3D localization, which makes it popular in modern vehicle localization algorithms.…”

Section: Introductionmentioning

confidence: 99%

CoFi: Coarse-to-Fine ICP for LiDAR Localization in an Efficient Long-lasting Point Cloud Map

Lyu¹,

Huang²,

Zhang³

2021

Preprint

View full text Add to dashboard Cite

LiDAR odometry and localization has attracted increasing research interest in recent years. In the existing works, iterative closest point (ICP) is widely used since it is precise and efficient. Due to its non-convexity and its local iterative strategy, however, ICP-based method easily falls into local optima, which in turn calls for a precise initialization. In this paper, we propose CoFi, a Coarse-to-Fine ICP algorithm for LiDAR localization. Specifically, the proposed algorithm downsamples the input point sets under multiple voxel resolution, and gradually refines the transformation from the coarse point sets to the fine-grained point sets. In addition, we propose a map based LiDAR localization algorithm that extracts semantic feature points from the LiDAR frames and apply CoFi to estimate the pose on an efficient point cloud map. With the help of the Cylinder3D algorithm for LiDAR scan semantic segmentation, the proposed CoFi localization algorithm demonstrates the state-of-the-art performance on the KITTI odometry benchmark, with significant improvement over the literature.

show abstract

RaLL: End-to-End Radar Localization on Lidar Map Using Differentiable Measurement Model

Cited by 23 publications

References 56 publications

Radar-to-Lidar: Heterogeneous Place Recognition via Joint Learning

Radar-to-Lidar: Heterogeneous Place Recognition via Joint Learning

Neural Motion Prediction for In-flight Uneven Object Catching

CoFi: Coarse-to-Fine ICP for LiDAR Localization in an Efficient Long-lasting Point Cloud Map

Contact Info

Product

Resources

About