The phonon damping model (PDM) is extended to include the effect of angular momentum at finite temperature. The model is applied to the study of damping of giant dipole resonance (GDR) in hot and noncollectively rotating spherical nuclei. The numerical results obtained for 88 Mo and 106 Sn show that the GDR width increases with both temperature T and angular momentum M .At T > 4 MeV and M ≤ 60 the increase in the GDR width slows down for 106 Sn, whereas at M ≥ 80 the GDR widths in both nuclei nearly saturate. By adopting the nuclear shear viscosity extracted from fission data at T = 0, it is shown that the maximal value of the angular momentum for 88 Mo and 106 Sn should be around 46 and 55 , respectively, so that the universal conjecture for the lower bound of the specific shear viscosity for all fluids is not violated up to T = 5 MeV.