Particles are spontaneously created from the vacuum by time-varying gravitational or electromagnetic backgrounds. It has been proven that the particle number operator in an expanding universe is an adiabatic invariant. In this paper we show that, in some special cases, the expected adiabatic invariance of the particle number fails in presence of electromagnetic backgrounds. Furthermore, we also show a close relation between the breaking of the adiabatic invariance and the emergence of the axial anomaly.