We study surface plasmons on two-dimensional square arrays of sub-wavelength holes in a gold film deposited on an optically-excited semiconductor. We observe four resonances of which we measure the resonance frequencies, the spectral widths, and the relative intensities. The spectral widths allow us to quantify various loss processes, including ohmic loss, optical absorption/gain and radiative scattering loss. Prominent kinks in the plasmon dispersion relation occur around the Rayleigh anomaly. A coupled mode model that includes a frequency dependent gain of the semiconductor reproduces the main features in the experimental data.