The objective of this research was to evaluate the impact of organic sources (animal manures vs. plant residues at the rate of 10 t ha−1 each) on the productivity of hybrid rice (Oryza sativa L.) production under different levels of phosphorus (0, 30, 60, and 90 kg P ha−1) fertilization. Two separate field experiments were conducted. In experiment (1), impact of three animal manures sources (cattle, sheep, and poultry manures) and P levels were studied along with one control plot (no animal manure and P applied) was investigated. In experiment (2), three plant residues sources (peach leaves, garlic residues, and wheat straw) and P levels were studied along with one control plot (no plant residues and P applied). Both the experiments were carried out on small land farmer field at District Swabi, Khyber Pakhtunkhwa Province (Northwest Pakistan) during summer 2015. The results revealed that in both experiments the control plot had significantly (p ≤ 0.05) less productivity than the average of all treated plots with organic sources and P level. The increase in P levels in both experiments (animal manure vs. plant residues) resulted in higher rice productivity (90 > 60 > 30 > 0 kg P ha−1). In the experiment under animal manures, application of poultry manure increased rice productivity as compared with sheep and cattle manures (poultry > sheep > cattle manures). In the experiment under plant residues, application of peach leaves or garlic residues had higher rice productivity than wheat straw (peach leaves = garlic residues > wheat straw). On average, rice grown under animal manures produced about 20% higher grain yield than rice grown under crop residues. We conclude from this study that application of 90 kg P ha−1 along with combined application of animal manures, especially poultry manure increases rice productivity. Also, the use of either garlic residues or peach leaves, never applied before as organic manures, can increase crop productivity and will help in degraded soil for sustainable soil management.