Intrusion detection systems can identify intrusion processes which are attempting to intrude, in the process of intruding, or have already occurred.Intrusion detection is a proactive defense approach. Intrusion detection system models built by machine learning are very sensitive to hyper-parameter settings, and different combinations of hyper-parameters can dramatically affect the model's capabilities. In previous work, finding hyperparameters corresponding to advanced models is a difficult task. In order to deal with the huge amount of network traffic data, we here propose the Gaussian Process Regression and Random Forest (GPR-RF) method, which uses Bayesian optimization of Gaussian process regression to find more appropriate combinations of Random Forest hyperparameters, and realizes the following two advantages: 1. The accuracy of the model is greatly improved compared to the traditional methods. 2. The method can quickly converge to better configurations. Among several methods compared, our method performs best in both aspects.