Background Understanding non-epidemiological factors is essential for the surveillance and prevention of infectious diseases, and the factors are likely to vary spatially and temporally as the disease progresses. However, the impacts of these influencing factors were primarily assumed to be stationary over time and space in the existing literature. The spatiotemporal impacts of mobility-related and social-demographic factors on disease dynamics remain to be explored. Methods Taking daily cases data during the coronavirus disease 2019 (COVID-19) outbreak in the US as a case study, we develop a mobility-augmented geographically and temporally weighted regression (M-GTWR) model to quantify the spatiotemporal impacts of social-demographic factors and human activities on the COVID-19 dynamics. Different from the base GTWR model, the proposed M-GTWR model incorporates a mobility-adjusted distance weight matrix where travel mobility is used in addition to the spatial adjacency to capture the correlations among local observations. Results The results reveal that the impacts of social-demographic and human activity variables present significant spatiotemporal heterogeneity. In particular, a 1% increase in population density may lead to 0.63% more daily cases, and a 1% increase in the mean commuting time may result in 0.22% increases in daily cases. Although increased human activities will, in general, intensify the disease outbreak, we report that the effects of grocery and pharmacy-related activities are insignificant in areas with high population density. And activities at the workplace and public transit are found to either increase or decrease the number of cases, depending on particular locations. Conclusions Through a mobility-augmented spatiotemporal modeling approach, we could quantify the time and space varying impacts of non-epidemiological factors on COVID-19 cases. The results suggest that the effects of population density, socio-demographic attributes, and travel-related attributes will differ significantly depending on the time of the pandemic and the underlying location. Moreover, policy restrictions on human contact are not universally effective in preventing the spread of diseases.