In this study, the rust conversion coating that acts directly on the surface of rust layer, was studied from the preparation of rust conversion agents gallic acid (GA), 2‐hydroxyphosphonyl acetic acid (HPAA), and their compound (GA/HPAA), rust conversion effect and anticorrosive mechanism. Results show that 1) when GA and HPAA were combined to form GA/HPAA with a mass ratio of 2:1, the strengthening effect of rust conversion was observed, and a more complete, dense, and smooth covering film was generated on the rust surface. In this case, the corrosion current of GA/HPAA was reduced to 4.10 × 10−7 A cm−2, while the GA was 2.35 × 10−6 A cm−2 and HPAA was 2.16 × 10−6 A cm−2. 2) The mechanism of rust conversion and corrosion resistance can be expressed as that the chelation reaction can occur between the rust and rust conversion agent, which can generate new coordination covalent bonds O–Fe–O, P–O–Fe, and C–Fe–O. In this case, they can firmly lock the rust and prevent it from spreading further. Meanwhile, a highly cross‐linked dense anticorrosion protective film can be formed. Due to the presence of these chelating products, they can act as obstacles, prolonging the diffusion path of corrosive medium and slowing down the interaction with the metal matrix.