Background: Attention-deficit/hyperactivity disorder (ADHD) and lower cognitive ability have been linked with increased likelihood of exposure to adversity. We hypothesized that these associations may be partly due to genetic factors. Methods: We calculated polygenic scores for ADHD and intelligence and assessed psychopathology and general cognitive ability in a sample of 297 youth aged 5-27 years enriched for offspring of parents with mood and psychotic disorders. We calculated an adversity score as a mean of 10 indicators, including socio-economic disadvantage, childhood maltreatment and bullying. We tested the effects of polygenic scores, externalizing symptoms and IQ on adversity scores using mixed-effects linear regression. Results: Externalizing symptoms and general cognitive ability showed expected positive and negative relationships with adversity, respectively. Polygenic scores for intelligence were unrelated to adversity, but polygenic scores for ADHD were associated with adversity (b = 0.23, 95% CI 0.13 to 0.34, p < .0001). ADHD polygenic scores uniquely explained 4.0% of variance in adversity score. The relationship between polygenic scores for ADHD and adversity was independently significant among individuals with (b = 0.49, 95% CI 0.25 to 0.75, p < .0001) and without (b = 0.14, 95% CI 0.02 to 0.26, p = .022) ADHD. Conclusions: A genetic score indexing liability to ADHD was associated with exposure to adversity in early life. Previously observed associations between externalizing symptoms, lower cognitive ability and adversity may be partially attributed to genetic liability to ADHD.