Retrograde vesicle trafficking pathways are responsible for returning membrane-associated components from endosomes to the Golgi apparatus and the endoplasmic reticulum (ER), and they are critical for maintaining organelle identity, lipid homeostasis, and many other cellular functions. The retrograde transport pathway has emerged as an important target for intravacuolar bacterial pathogens. The opportunistic pathogen exploits both the secretory and recycling branches of the vesicle transport pathway for intracellular bacterial proliferation. Its Dot/Icm effector RidL inhibits the activity of the retromer by directly engaging retromer components. However, the mechanism underlying such inhibition remains unknown. Here we present the crystal structure of RidL in complex with VPS29, a subunit of the retromer. Our results demonstrate that RidL binds to a highly conserved hydrophobic pocket of VPS29. This interaction is critical for endosomal recruitment of RidL and for its inhibitory effects. RidL inhibits retromer activity by direct competition, in which it occupies the VPS29-binding site of the essential retromer regulator TBC1d5. The mechanism of retromer inhibition by RidL reveals a hotspot on VPS29 critical for recognition by its regulators that is also exploited by pathogens, and provides a structural basis for the development of small molecule inhibitors against the retromer.