Small ponds—farm ponds, detention ponds, or impoundments below 0.01 km2—serve important human needs throughout most large river basins. Yet the role of small ponds in regional nutrient and sediment budgets is essentially unknown, currently making it impossible to evaluate their management potential to achieve water quality objectives. Here we used new hydrography data sets and found that small ponds, depending on their spatial position within both their local catchments and the larger river network, can dominate the retention of nitrogen, phosphorus, and sediment compared to rivers, lakes, and reservoirs. Over 300,000 small ponds are collectively responsible for 34%, 69%, and 12% of the mean annual retention of nitrogen, phosphorus, and sediment in the Northeastern United States, respectively, with a dominant influence in headwater catchments (54%, 85%, and 50%, respectively). Small ponds play a critical role among the many aquatic features in long‐term nutrient and sediment loading to downstream waters.