According to the maximum entropy principle, it has been proved that the gravitational field equations could be derived by the extrema of the total entropy for a perfect fluid, which implies that thermodynamic relations contain information as regards gravity. In this manuscript, we obtain a criterion for the thermodynamical stability of an adiabatic, self-gravitating perfect fluid system by the second variation of the total entropy. We show, for Einstein's gravity with spherical symmetry spacetime, that the criterion is consistent with that for the dynamical stability derived by Chandrasekhar and Wald. We also find that the criterion could be applied to cases without spherical symmetry, or under general perturbations. The result further establishes the connection between thermodynamics and gravity.