We give some conditions on a family of abelian covers of P 1 of genus g curves, that ensure that the family yields a subvariety of Ag which is not totally geodesic, hence it is not Shimura. As a consequence, we show that for any abelian group G, there exists an integer M which only depends on G such that if g > M , then the family yields a subvariety of Ag which is not totally geodesic. We prove then analogous results for families of abelian covers of Ct → P 1 = Ct/ G with an abelian Galois group G of even order, proving that under some conditions, if σ ∈ G is an involution, the family of Pryms associated with the covers Ct → Ct = Ct/ σ yields a subvariety of A δ p which is not totally geodesic. As a consequence, we show that if G = (Z/N Z) m with N even, and σ is an involution in G, there exists an integer M (N ) which only depends on N such that, if g = g( Ct) > M (N ), then the subvariety of the Prym locus in A δ p induced by any such family is not totally geodesic (hence it is not Shimura).