Extinction spectra of single gold nanorods optically trapped in water were measured by spatial modulation spectroscopy. Comparison of the extinction cross sections and resonance frequencies to finite element calculations allows us to determine the dimensions of the nanorod and estimate the contribution of radiation damping to the LSPR line width. Subtracting the radiation damping and bulk contributions from the measured line widths yields the electron-surface scattering contribution. The results show that the surfactant coating for the nanorods causes large electron-surface scattering effects with significant particle-to-particle variations. These effects are more pronounced than those seen for substrate-supported particles in previous single particle studies. Indeed, the measured line widths are only slightly narrower than that of the ensemble spectrum. These results show the importance of removing surfactant for sensing applications of these materials.