Considering time-varying meshing stiffness, gear backlash, static transmission error and tooth face friction, a nonlinear dynamic model for a spur gear pair is proposed to research systematically the dynamic behaviors of system, in which the meshing stiffness of gear pair is deduced and calculated in terms of the extending period method. Meanwhile, the sliding friction force under single-tooth and double-tooth meshing regions is constructed as a function of the meshing principle. Based on the developed model, the bifurcation and chaos characteristics of system under lightly and heavily loaded conditions are studied, respectively, by applied Runge-Kutta numerical method, and the parametric effects of rotational speed, damping ratio and gear backlash on the dynamic behaviors are investigated detailedly. Bifurcation diagram, three-dimensional frequency spectrum, time-domain waveform, frequency plot, phase diagram, Poincaré map and dynamic load are used to discuss and determine motion states and dynamic responses of system. The numerical results represent that with the change of control parameters the system undergoes various types of motion states under different loaded conditions. The corresponding meshing contact states of tooth pair are transformed among no impact, single-sided impact and double-sided impact. The research results can provide certain guidance for choosing suitable parameter values to reduce the amplitude of vibration and even avoid the chaotic response in gear system.