The interface between the vapor and liquid phase of quadrupolar-dipolar fluids is the seat of an electric interfacial potential whose influence on ion solvation and distribution is not yet fully understood. To obtain further microscopic insight into water specificity we first present extensive classical molecular dynamics simulations of a series of model liquids with variable molecular quadrupole moments that interpolates between SPC/E water and a purely dipolar liquid. We then pinpoint the essential role played by the competing multipolar contributions to the vapor-liquid and the solute-liquid interface potentials in determining an important ion-specific direct electrostatic contribution to the ionic solvation free energy for SPC/E water-dominated by the quadrupolar and dipolar parts-beyond the dominant polarization one. Our results show that the influence of the vapor-liquid interfacial potential on ion solvation is strongly reduced due to the strong partial cancellation brought about by the competing solute-liquid interface potential.