We study the decoherence process associated with the scattering of stochastic backgrounds of gravitational waves. We show that it has a negligible influence on HYPER-like atomic interferometers although it may dominate decoherence of macroscopic motions, such as the planetary motion of the Moon around the Earth. *