Background Pregnant women may be especially susceptible to negative events (i.e. adversity) related to the coronavirus disease 2019 (COVID-19) pandemic and negative affective responses to these events (i.e. stress). We examined the latent structure of stress and adversity related to the COVID-19 pandemic among pregnant women, potential antecedents of COVID-19-related stress and adversity in this population, and associations with prenatal depressive symptoms. Method We surveyed 725 pregnant women residing in the San Francisco Bay Area in March−May 2020, 343 of whom provided addresses that were geocoded and matched by census tract to measures of community-level risk. We compared their self-reported depressive symptoms to women matched on demographic factors and history of mental health difficulties who were pregnant prior to the pandemic. Results Women who were pregnant during the pandemic were nearly twice as likely to have possible depression than were matched women who were pregnant prior to the pandemic. Individual- and community-level factors tied to socioeconomic inequality were associated with latent factors of COVID-19-related stress and adversity. Beyond objective adversity, subjective stress responses were strongly associated with depressive symptoms during the pandemic. Conclusions Highlighting the role of subjective responses in vulnerability to prenatal depression and factors that influence susceptibility to COVID-19-related stress, these findings inform the allocation of resources to support recovery from this pandemic and future disease outbreaks. In addition to policies that mitigate disruptions to the environment due to the pandemic, treatments that focus on cognitions about the self and the environment may help to alleviate depressive symptoms in pregnant women.