The leaching of soil nitrogen (N) has become one of the most concerning environmental threats to ecosystems. Arbuscular mycorrhizal (AM) fungi have important ecological functions, however, their influence on soil N leaching and the mechanism of action remain unclear. We conducted a two-factor (N application level × AM inoculation) experiment on poplar, and for the first time, comprehensively analyzed the mechanism by which AM fungi influence soil N leaching. The results showed that, under optimum (7.5 mM) and high (20 mM) N levels, the nitrate (NO3−) and ammonium (NH4+) concentrations of leachate in the AM inoculated treatment (+AM) were lower than in the non-inoculated treatment (−AM), with significant reductions of 20.0% and 67.5%, respectively, under high N level, indicating that AM inoculation can reduce soil N leaching and that it is more effective for NH4+. The arbuscular and total colonization rates gradually increased, and the morphology of spores and vesicles changed as the N level increased. Under optimum and high N levels, +AM treatment increased the root N concentration by 11.7% and 50.7%, respectively; the increase was significant (p < 0.05) at the high N level, which was associated with slightly increased transpiration and root activity despite reductions in root surface area and root length. Additionally, the +AM treatment increased soil cation exchange capacity (CEC), soil organic carbon (SOC), and significantly (p < 0.05) increased the proportions of macroaggregates (but without significant change in microaggregates), causing soil total nitrogen (TN) to increase by 7.2% and 4.7% under optimum and high N levels, respectively. As the N levels increased, the relative contributions of AM inoculation on N leaching increased, however, the contributions of plant physiological and soil variables decreased. Among all of the variables, SOC had important contributions to NH4+ and total N in the leachate, while root N concentration had a higher contribution to NO3−. In conclusion, AM fungi can mitigate soil N leaching and lower the risk of environmental pollution via enhancing N interception by the inoculated fungi, increasing N sequestration in plant roots, and by improving soil N retention.