Phase shift error and contrast variation caused by vibration lead to a large measurement error in temporal phase-shifting interferometry (PSI). To suppress the error, a sideband correlation algorithm is proposed to detect phase shift and contrast variation. The tilt factors and translational values of phase shift are determined by analyzing the correlations of spectral sidebands of interferograms. The contrast variations are determined by detecting the modulus of the baseband and sideband correlation result. A least-squares equation with contrast compensation is established to retrieve the wavefront phase. The algorithm requires a set of temporal phase-shifting interferograms, each one also containing a moderate amount of spatial-carrier; the interferograms may have an arbitrary aperture. Simulations demonstrate the reliability, and the experiments under vibration show the practical effectiveness of the algorithm.