Our aim is to assess how environmental and social variables are associated with COVID-19 mortality among countries and in the first and second waves among Brazilian states. We show that social isolation was not significantly, or was positively associated with COVID-19 mortality, which probably reflects people staying at home in the high mortality periods. The magnitude of temperature effects in Brazil varied depending on whether or not Index of Human Development was included in the regression analyses, but higher temperatures consistently reduced per capita deaths among countries. Availability of hospital bed capacity or intensive-care units had no detectable effect on mortality among countries, and within Brazil there was a positive relationship with excess deaths and a negative relationship with deaths per case during the second wave. Counterintuitively, mean age was negatively associated with deaths among Brazilian states, but was positively associated with mortality among countries. Mortality tended to be higher in states and countries with greater urbanization. Overall, the relationships of mortality-curve flatness, social isolation, temperature and hospital infrastructure with COVID-19 mortality were much weaker than is often assumed or are opposite in sign to the predictions, indicating that more complex models are needed to confront future epidemics.