Grassland gross primary productivity (GPP) is an important part of global terrestrial carbon flux, and its accurate simulation and future prediction play an important role in understanding the ecosystem carbon cycle. Machine learning has potential in large-scale GPP prediction, but its application accuracy and impact factors still need further research. This paper takes the Mongolian Plateau as the research area. Six machine learning methods (multilayer perception, random forest, Adaboost, gradient boosting decision tree, XGBoost, LightGBM) were trained using remote sensing data (MODIS GPP) and 14 impact factor data and carried out the prediction of grassland GPP. Then, using flux observation data (positions of flux stations) and remote sensing data (positions of non-flux stations) as reference data, detailed accuracy evaluation and comprehensive trade-offs are carried out on the results, and key factors affecting prediction performance are further explored. The results show that: (1) The prediction results of the six methods are highly consistent with the change tendency of the reference data, demonstrating the applicability of machine learning in GPP prediction. (2) LightGBM has the best overall performance, with small absolute error (mean absolute error less than 1.3), low degree of deviation (root mean square error less than 3.2), strong model reliability (relative percentage difference more than 5.9), and a high degree of fit with reference data (regression determination coefficient more than 0.97), and the prediction results are closest to the reference data (mean bias is only −0.034). (3) Enhanced vegetation index, normalized difference vegetation index, precipitation, land use/land cover, maximum air temperature, potential evapotranspiration, and evapotranspiration are significantly higher than other factors as determining factors, and the total contribution ratio to the prediction accuracy exceeds 95%. They are the main factors influencing GPP prediction. This study can provide a reference for the application of machine learning in GPP prediction and also support the research of large-scale GPP prediction.