This article focuses on the way that staff and guardians in the rural Nottinghamshire workhouse of Southwell sought to exert control and containment over pauper inmates. Fusing together local and central records for the period 1834–71, including locally held punishment books and correspondence at The National Archives, Kew (TNA), we argue that the notional power of the workhouse authorities was heavily shaded. Most paupers most of the time did not find their behaviour heavily and clumsily controlled. Rather, staff focused their attention in terms of detecting and punishing disorderly behaviour on a small group of long-term and often mentally ill paupers whose actions might create enmities or spiral into larger conflicts and dissent in the workhouse setting. Both inmates and those under threat of workhouse admission would have seen or heard about punishment of ‘the usual characters’. This has important implications for how we understand the intent and experience of the New Poor Law up to the formation of the Local Government Board (LGB) in 1871.