In this paper, we consider a smooth connected finite-dimensional manifold M, an affine connection ∇ with holonomy group H ∇ and a smooth completely non integrable distribution. We define the -horizontal holonomy group H ∇ as the subgroup of H ∇ obtained by ∇-parallel transporting frames only along loops tangent to . We first set elementary properties of H ∇ and show how to study it using the rolling formalism Chitour and Kokkonen (2011). In particular, it is shown that H ∇ is a Lie group. Moreover, we study an explicit example where M is a free step-two homogeneous Carnot group with m ≥ 2 4 Helsinki, Finland Boutheina Hafassa et al.generators, and ∇ is the Levi-Civita connection associated to a Riemannian metric on M, and show in this particular case that H ∇ is compact and strictly included in H ∇ as soon as m ≥ 3.