A particle with finite initial velocity in a disordered potential comes back and in average stops at the original location. This phenomenon dubbed 'quantum boomerang effect' (QBE) has been recently observed in an experiment simulating the quantum kicked-rotor model [Phys. Rev. X 12, 011035 (2022)]. We provide analytical arguments that support QBE in a wide class of disordered systems. Sufficient conditions to observe the real-space QBE effect are (a) Anderson localization, (b) the reality of the spectrum for the case of non-Hermitian systems, (c) the ensemble of disorder realizations {H} be invariant under the application of R T , and (d) the initial state is an eigenvector of R T , where R is a reflection x → −x and T is the time-reversal operator. The QBE can be observed in momentum-space in systems with dynamical localization if conditions (c) and (d) are satisfied with respect to the operator T instead of RT . These conditions allow the observation of the QBE in time-reversal symmetry broken models, contrarily to what was expected from previous analyses of the effect, and to a large class of non-Hermitian models. We provide examples of QBE in ladder models with magnetic flux breaking time-reversal symmetry and in a non-Hermitian random-hopping model. Whereas the QBE straightforwardly applies to noninteracting many-body systems, we argue that breaking of RT (T ) symmetry is responsible for the absence of real-(momentum-)space QBE in weakly interacting bosonic systems.