Despite of the widespread use of the neural networks in the industrial applications, their mathematical formulation remains difficult to analyze. This explains a limited amount of work that formally models their classification volatility. Referring to the statistical point of view, we attempt in this work to evaluate the classical and Bayesian neural networks stability degree compared to the statistical methods stressing their error rate probability densities. The comparison based on this new criterion is performed using the modified semi-bounded Plug-in algorithm.