Under the influence of climate and human factors, runoff and sediment in the Yellow River Basin (YRB) have undergone a significant decline in the past decades. Based on the meteorological and hydrological records of six hydrological stations in the mainstream from 1967 to 2015, the impacts of climate change and human activities on the changes of runoff and sediment in different river reaches of the YRB were quantitatively evaluated by applying the Budyko framework. The results indicated that (1) the precipitation and potential evaporation suggested a non-significant decreasing trend in the whole basin (P > 0.05); (2) the contributions of climate change and human activities to the reduction of runoff in the basins of the six hydrological stations ranged from 7.6 to 19.7% and from 80.3 to 92.4%, respectively, with the largest contribution of human activities in the middle reaches; (3) the contributions of climate change and human activities to the reduction of sediment in the catchments ranged from −0.3 to 9.4% and from 90.6 to 100.3%, respectively, and the most artificial efforts have been made to reduce sediment in the middle reaches on account of mainly sediment source.