Soil erosion has become a major global environmental problem and is particularly acute on the Loess Plateau (LP), China. It is therefore highly important to control this process in order to improve ecosystems, protect ecological security, and maintain the harmonious relationship between humans and nature. We compared the effects of rainfall and land use (LU) patterns on soil erosion in different LP watersheds in this study in order to augment and improve soil erosion models. As most research on this theme has so far been focused on individual study areas, limited analyses of rainfall and LU patterns on soil erosion within different-scale watersheds has so far been performed, a discrepancy which might influence the simulation accuracies of soil erosion models. We therefore developed rainfall and LU pattern indices in this study using the soil erosion evaluation index as a reference and applied them to predict the extent of this process in different-scale watersheds, an approach which is likely to play a crucial role in enabling the comprehensive management of this phenomenon as well as the optimized design of LU patterns. The areas considered in this study included the Qingjian, Fenchuan, Yanhe, and Dali river watersheds. Results showed that the rainfall erosivity factor (R) tended to increase in these areas from 2006 to 2012, while the vegetation cover and management factor (C) tended to decrease. Results showed that as watershed area increased, the effect of rainfall pattern on soil erosion gradually decreased while patterns in LU trended in the opposite direction, as the relative proportion of woodland decreased and the different forms of steep slope vegetation cover became more homogenous. As watershed area increased, loose soil and craggy terrain properties led to additional gravitational erosion and enhanced the effects of both soil and topography.