Multi-temporal interferometric synthetic aperture radar (MT-InSAR) can be applied to monitor the structural health of infrastructure such as railways, bridges, and highways. However, for the successful interpretation of the observed deformation within a structure, or between structures, it is imperative to associate a radar scatterer unambiguously with an actual physical object. Unfortunately, the limited positioning accuracy of the radar scatterers hampers this attribution, which limits the applicability of MT-InSAR. In this study, we propose an approach for health monitoring of railway system combining MT-InSAR and LiDAR (laser scanning) data. An amplitude-augmented interferometric processing approach is applied to extract continuously coherent scatterers (CCS) and temporary coherent scatterers (TCS), and estimate the parameters of interest. Based on the 3D confidence ellipsoid and a decorrelation transformation, all radar scatterers are linked to points in the point cloud and their coordinates are corrected as well. Additionally, several quality metrics defined using both the covariance matrix and the radar geometry are introduced to evaluate the results. Experimental results show that most radar scatterers match well with laser points and that LiDAR data are valuable as auxiliary data to classify the radar scatterers.