SU(N) Lie algebras possess discrete symmetries which can lead naturally to stable vector dark matter (DM). In this work, we consider the possibility that the dark SU(N) sector couples to the visible sector through the Higgs portal. We find that minimal CP-conserving hidden "Higgs sectors" entail stable massive gauge fields which fall into the WIMP category of dark matter candidates. For SU(N), N > 2, DM consists of three components, two of which are degenerate in mass. In all of the cases, there are substantial regions of parameter space where the direct and indirect detection as well as relic abundance constraints are satisfied.