Energy transfer from plasmonic nanoparticles to semiconductors can expand the available spectrum of solar energy-harvesting devices. Here, we spatially and spectrally resolve the interaction between single Ag nanocubes with insulating and semiconducting substrates using electron energy-loss spectroscopy, electrodynamics simulations, and extended plasmon hybridization theory. Our results illustrate a new way to characterize plasmon-semiconductor energy transfer at the nanoscale and bear impact upon the design of next-generation solar energy-harvesting devices.