The impact of variable infection risk by race and ethnicity on the dynamics of SARS-CoV-2 spread is largely unknown. Here, we fit structured compartmental models to seroprevalence data from New York State and analyze how herd immunity thresholds (HITs), final sizes, and epidemic risk changes across racial and ethnic groups. A proportionate mixing model reduced the overall HIT, but more realistic levels of assortative mixing increased the threshold. Across all models, the burden of infection fell disproportionately on minority populations: in an assortative mixing model fit to Long Island census data, 80% of Hispanics or Latinos were infected when the HIT is reached compared to 33% of non-Hispanic whites. Our findings, which are meant to be illustrative and not best estimates, demonstrate how racial and ethnic disparities can impact epidemic trajectories and result in a dis-proportionate distribution of the burden of SARS-CoV-2 infection.