We study the wave propagation modes in the relativistic streaming pair plasma of the magnetospheres of pulsars and magnetars, focusing on the effect of vacuum polarization. We show that the combined plasma and vacuum polarization effects give rise to a vacuum resonance, where ‘avoided mode crossing’ occurs between the extraordinary mode and the (superluminous) ordinary mode. When a photon propagates from the vacuum‐polarization‐dominated region at small radii to the plasma‐dominated region at large radii, its polarization state may undergo significant change across the vacuum resonance. We map out the parameter regimes (e.g. field strength, plasma density and Lorentz factor) under which the vacuum resonance occurs and examine how wave propagation is affected by the resonance. Some possible applications of our results are discussed, including high‐frequency radio emission from pulsars and possibly magnetars, and optical/infrared emission from neutron star surfaces and inner magnetospheres.