We investigate the nightmare scenario of dark sectors that are made of non-abelian gauge theories with fermions, gravitationally coupled to the Standard Model (SM). While testing these scenarios is experimentally challenging, they are strongly motivated by the accidental stability of dark baryons and pions, that explain the cosmological stability of dark matter (DM). We study the production of these sectors which are minimally populated through gravitational freeze-in, leading to a dark sector temperature much lower than the SM, or through inflaton decay, or renormalizable interactions producing warmer DM. Despite having only gravitational couplings with the SM these scenarios turn out to be rather predictive depending roughly on three parameters: the dark sector temperature, the confinement scale and the dark pion mass. In particular, when the initial temperature is comparable to the SM one these scenarios are very constrained by structure formation, ∆Neff and limits on DM self-interactions. Dark sectors with same temperature or warmer than SM are typically excluded.