Background: Structural variation in subcortical brain regions has been linked to substance use, including the most prevalent substances nicotine and alcohol. It may be that pre-existing differences in subcortical brain volume affect smoking and alcohol use, but there is also evidence that smoking and alcohol use can lead to structural changes. We assess the causal nature of this complex relationship with bi-directional Mendelian randomization (MR). Methods: MR uses genetic variants predictive of a certain trait (exposure) as instrumental variables to test causal effects on a certain outcome. Due to random assortment at meiosis, genetic variants shouldnt be associated with confounders, allowing less biased causal inference. We employed summary-level data of the largest available genome-wide association studies of subcortical brain region volumes (nucleus accumbens, amygdala, caudate nucleus, hippocampus, pallidum, putamen, and thalamus; n=50,290) and smoking and alcohol use (smoking initiation, n=848,460; cigarettes per day, n=216,590; smoking cessation, n=378,249; alcohol drinks per week, n=630,154; alcohol dependence, n=46,568). The main analysis, inverse-variance weighted regression, was verified by a wide range of sensitivity methods. Results: There was strong evidence that alcohol dependence decreased amygdala and hippocampal volume and that smoking more cigarettes per day decreased hippocampal volume. From subcortical brain volumes to substance use, there was no or weak evidence for causal effects. Conclusions: Our findings suggest that heavy alcohol use and smoking can causally reduce subcortical brain volume. This adds to accumulating evidence that alcohol and smoking affect the brain, and most likely mental health, warranting more recognition in public health efforts.