The damping performance of lightweight steel/polymer/steel sandwich composites has been typically studied without considering the interface between the heterogeneous materials. Herein, the interface between the steel and polymer layers was investigated experimentally for various steels and polymers. Impact hammer tests revealed that the interfacial adhesion strength between steel and polymer did not significantly affect the loss factor of the sandwich cantilever beam composites. However, although very thin, the adhesive layer itself greatly influenced the vibration-damping performance of the composite. As such, previous three-layered (skin-core-skin) sandwich model without considering the properties of adhesive layer predicted different results from experiments. To accurately predict the loss factor of the sandwich, a five-layered sandwich model consisting of two skins, two adhesives, and a core layer was proposed and validated with experiments.