The social history of convicts is an area of study which has hitherto remained an uncharted territory. The muted convict voice makes an ephemeral appearance in most colonial histories as ‘convict resistance’, which is seen as the sum total of the convict's life experience. On the other hand, colonial records and monographs oscillate between two extremes in the categorization of the convict's social life. There is either a romanticization of the idyllic penal colonies where the convicts have the appearance of reformed savages, peacefully going about their daily chores, no different from the Indian peasants; or the convicts are seen as conniving brutes obsessed with the idea of ‘escape’, where the state of unfreedom that they are subjected to is seen as the most defining characteristic of their life in the penal settlement. Breaking out of these moulds, this essay has used the colonial project of settling the convict in the penal colony on the Andaman Islands as an entry point into the social life of the convict. The convict, when he arrived in the Andamans was akin to a shipwrecked vessel, severed from all ties and floating without an anchor. The doors of the social world he had inhabited before he set foot on the Andamans were supposed to be closed to him forever. The essay examines the attempt of the state to rebuild the life of such a dispossessed and beleaguered social being. The main denominators of the domestication of the convict were matrimony and setting up of a household. The essay explores the different notions held by the convicts and the state with regard to ‘family’ and ‘marriage’, and the fissures and tensions created in the system thereby. It draws upon government records, colonial monographs, interviews with descendants of convicts, and leaflets out of the lives of the convicts to narrate a social history of the Settlement.