Simulation of surface erosion by impact of abrasive water-jet (AWJ) is challenging for traditional mesh-based numerical methods, because it involves complex phenomena related to fluid–solid interaction, material removal and free surface flows. In this study, the surface erosion mechanism of ductile target materials by AWJ is investigated based on the smoothed particle hydrodynamics method (SPH, a mesh-free method). The water-jet, abrasives and solid target are all discretized with a series of SPH particles. The water-jet is modeled as a continuous fluid flow, the target material is modeled as a elastic–plastic material, and the abrasives are modeled as rigid bodies. The SPH model is improved by combining several correction algorithms and techniques, which help us to increase the stability and the accuracy of the simulation. The density diffusion correction term is introduced for water-jet, which reduces the pressure noise and makes the pressure field more stable. The dynamic boundary algorithm is used for the abrasives, and the pressure distribution around the abrasive is improved. The interactions among the water-jet, abrasives and solid target are realized by different contact algorithms. The dynamic process of plastic deformation, material removal and crater generation of the target material under AWJ impact can be simulated by the improved SPH model. The proposed model could be useful in the applications of AWJ machining and metal surface erosion.