We investigate the instabilities that may lead to the breakdown of the Luttinger liquid in the small-diameter ͑5,0͒ nanotubes, paying attention to the competition between the effective interaction mediated by phonon exchange and the Coulomb interaction. We use bosonization methods to achieve an exact treatment of the Coulomb interaction at small momentum transfer, and apply next renormalization group methods to analyze the low-energy behavior of the electron system. This allows us to discern the growth of several response functions for charge-density-wave modulations and for superconducting instabilities with different order parameters. We show that, in the case of single nanotubes exposed to screening by external gates, the Luttinger liquid is unstable against the onset of a strong-coupling phase with very large charge-density-wave correlations. The temperature of crossover to the new phase depends crucially on the dielectric constant of the environment, ranging from T c ϳ 10 −4 K ͑at Ϸ 1͒ up to a value T c ϳ 10 2 K ͑reached from Ϸ 10͒. The physical picture is however different when we consider the case of a large array of nanotubes, in which there is a three-dimensional screening of the Coulomb interaction over distances much larger than the intertube separation. The electronic instability is then triggered by the divergence of one of the charge stiffnesses in the Luttinger liquid, implying a divergent compressibility and the appearance of a regime of phase separation into spatial regions with excess and defect of electron density.