We investigate the viscous oscillatory instability (overstability) of an unperturbed dense planetary ring, an instability that might play a role in the formation of radial structure in Saturn's B-ring. We generalize existing hydrodynamic models by including the heat flow equation in the analysis and compare our results to the development of overstable modes in local particle simulations. With the heat flow, in addition to the balance equations for mass and momentum, we take into account the balance law for the energy of the random motion; i.e., we allow for a thermal mode in a stability analysis of the stationary Keplerian flow. We also incorporate the effects of nonlocal transport of momentum and energy on the stability of the ring. In a companion paper (Salo, H., J. Schmidt, and F. Spahn 2001. Icarus, doi:10.1006/icar.2001.6680) we describe the determination of the local and nonlocal parts of the viscosity, the heat conductivity, the pressure, as well as the collisional cooling, together with their dependences on temperature and density, in local event-driven simulations of a planetary ring. The ring's self-gravity is taken into account in these simulations by an enhancement of the frequency of vertical oscillations z > . We use these values as parameters in our hydrodynamic model for the comparison to overstability in simulated rings of meter-sized inelastic particles of large optical depth with z / = 3.6. We find that the inclusion of the energy-balance equation has a stabilizing influence on the overstable modes, shifting the stability boundary to higher optical depths, and moderating the growth rates of the instability, as compared to a purely isothermal treatment. The non-isothermal model predicts correctly the growth rates and oscillation frequencies of overstable modes in the simulations, as well as the phase shifts and relative amplitudes of the perturbations in density and radial and tangential velocity.