This paper proposes a novel damage index for railway bridges based on synchronous strain and displacement data collected at the passage of trains. The approach identifies a transformation operator that converts strains into displacements in a data‐driven fashion without prior structural knowledge and with no parameter selection. The displacement prediction error is proposed as a robust damage index, insensitive to the vehicle loads and temperature effects. Numerical simulations and data generated through the calibrated finite element model of a steel truss bridge showcase the effectiveness of the proposed approach.