In this paper, we study the stochastic P-bifurcation problem for an axially moving bistable viscoelastic beam with fractional derivatives of high-order nonlinear terms under colored noise excitation. Firstly, using the principle for minimal mean square error, we show that the fractional derivative term is equivalent to a linear combination of the damping force and restoring force, so that the original system can be simplified to an equivalent system. Secondly, we obtain the stationary probability density function of the system amplitude by the stochastic averaging and the singularity theory, we find the critical parametric conditions for stochastic P-bifurcation of the system amplitude. Finally, we analyze different types of the stationary probability density function curves of the system qualitatively by choosing parameters corresponding to each region divided by the transition set curves. We verify the theoretical analysis and calculation of the transition set by showing the consistency of the numerical results obtained by Monte Carlo simulation with the analytical results. The method used in this paper directly guides the design of the fractional order viscoelastic material model to adjust the response of the system.