We investigate about Hardy-Rellich inequalities for perturbed Laplacians. In particular, we show that a non-trivial angular perturbation of the free operator typically improves the inequality, and may also provide an estimate which does not hold in the free case. The main examples are related to the introduction of a magnetic field: this is a manifestation of the diamagnetic phenomenon, which has been observed by Laptev and Weidl in [43] for the Hardy inequality, later by Evans and Lewis in [15] for the Rellich inequality, while for the so called Hardy-Rellich inequality has not been investigated yet, at our knowledge. After showing the optimal inequality, we prove that the best constant is not attained by any function in the domain of the estimate.