We perform a comprehensive analysis of the redshift evolution of the rest-UV spectra of star-forming galaxies out to z ∼ 5. We combine new z ∼ 5 measurements of HI Lyα and low-and high-ionization interstellar metal absorption features with comparable measurements at z ∼ 2-4. This redshift range covers the peak epoch of star formation in the Universe and extends back towards the epoch of reionization. We measure the equivalent widths of interstellar absorption features using stacked spectra in bins of Lyα equivalent width, and perform corrections to the strength of Lyα based on a model for the transmission of the intergalactic medium. We find a strong trend of decreasing low-ionization line strength with increasing Lyα emission strength over the redshift range z ∼ 2-5, suggesting that both of these quantities are fundamentally linked to neutral gas covering fraction. At the highest Lyα equivalent widths, we see evolution with increasing redshift towards greater Lyα emission strength at fixed low-ionization absorption strength. This evolution suggests a higher intrinsic production rate of Lyα photons at z ∼ 5 than at lower redshift. Our conclusion is supported by the joint evolution of the relationships among Lyα emission strength, interstellar absorption strength, and dust reddening. We perform additional analysis in bins of stellar mass, star-formation rate, UV luminosity, and age, examining how the relationships between galaxy properties and Lyα emission evolve towards higher redshift. We conclude that increasing intrinsic Lyα photon production and strong detection of nebular C IV emission (signaling lower metallicity) at z ∼ 5 indicate an elevated ionized photon production efficiency (ξ ion ).