The interfacial thermal conductance (ITC) quantifies the heat transport across material-fluid interfaces. It is a property of crucial importance to study heat transfer processes both at macro and nanoscales. Therefore, it is essential to accurately model the specific interactions between solids and liquids. Here we investigate the thermal conductance of gold-water interfaces using polarizable and non-polarizable models. Both models have been fitted to reproduce the interfacial tension of the gold-water interface, but they predict significantly different ITCs. We demonstrate that the treatment of polarization using Drude-like models, widely employed in molecular simulations, leads to a coupling of the solid and liquid's vibrational modes that gives rise to a significant overestimation of the ITCs. We analyze the dependence of the vibrational coupling with the mass of the Drude particle and propose a solution to the artificial enhancement of the ITC, preserving at the same time the polarization response of the solid. Based on our calculations, we estimate ITCs of 200 MW/(m2 K) for the water-gold interface. This magnitude is comparable to that reported recently for gold-water interfaces (279+-16 MW/(m2 K)), using atomic fluctuating charges to account for the polarization contribution.