Oversight, scholars argue, allows Congress to control the executive agents it empowers to implement law. Yet the tools of oversight are rather limited and debate continues as to how much influence oversight provides. How well can members of Congress motivate performance within the bureaucracy? To measure the efficacy of oversight, we create a new data set on a bureaucratic deficiency that Congress has sought to reduce since the early 2000s: improperly-made payments to contractors and clients. We estimate the effect of congressional hearings, one of the most important tools of congressional oversight, as well as correspondence, appropriation committee reports, statutes, and executive action. We find small or no effects of these tools on subsequent improper payments, suggesting strong limits on the efficacy of congressional oversight. Our findings imply that America's elected officials struggle to effectively manage implementation of government policy.