Optical properties of polar and nonpolar nitride quantum dots (QDs) are determined on the basis of a microscopic theory which combines a continuum elasticity approach to the polarization potential, a tight-binding model for the electronic energies and wavefunctions, and a many-body theory for the optical properties. For nonpolar nitride quantum dots, we find that optical absorption and emission spectra exhibit a weak ground-state oscillator strength in a single-particle calculation whereas the Coulomb configuration interaction strongly enhances the ground-state transitions. This finding sheds new light on existing discrepancies between previous theoretical and experimental results for these systems, as a weak ground state transition was predicted because of the spatial separation of the corresponding electron and hole state due to intrinsic fields whereas experimentally fast optical transitions have been observed.