The COVID-19 pandemic has yielded disproportionate impacts on communities of color in New York City (NYC). Researchers have noted that social disadvantage may result in limited capacity to socially distance, and consequent disparities. Here, we investigate the role of neighborhood social disadvantage on the ability to socially distance, infections, and mortality. We combine Census Bureau and NYC open data with SARS-CoV-2 testing data using supervised dimensionality-reduction with Bayesian Weighted Quantile Sums regression. The result is a ZIP code-level index with relative weights for social factors facilitating infection risk. We find a positive association between neighborhood social disadvantage and infections, adjusting for the number of tests administered. Neighborhood infection risk is also associated with capacity to socially isolate, as measured by NYC subway data. Finally, infection risk is associated with COVID-19-related mortality. These analyses support that differences in capacity to socially isolate is a credible pathway between disadvantage and COVID-19 disparities.