Background Mental burden due to the SARS-CoV-2 pandemic has been widely reported for the general public and specific risk groups like healthcare workers and different patient populations. We aimed to assess its impact on mental health during the early phase by comparing pandemic with prepandemic data and to identify potential risk and protective factors. Methods For this systematic review and meta-analyses, we systematically searched PubMed, PsycINFO, and Web of Science from January 1, 2019 to May 29, 2020, and screened reference lists of included studies. In addition, we searched PubMed and PsycINFO for prepandemic comparative data. Survey studies assessing mental burden by the SARS-CoV-2 pandemic in the general population, healthcare workers, or any patients (eg, COVID-19 patients), with a broad range of eligible mental health outcomes, and matching studies evaluating prepandemic comparative data in the same population (if available) were included. We used multilevel meta-analyses for main, subgroup, and sensitivity analyses, focusing on (perceived) stress, symptoms of anxiety and depression, and sleep-related symptoms as primary outcomes. Results Of 2429 records retrieved, 104 were included in the review (n = 208,261 participants), 43 in the meta-analysis (n = 71,613 participants). While symptoms of anxiety (standardized mean difference [SMD] 0.40; 95% CI 0.15–0.65) and depression (SMD 0.67; 95% CI 0.07–1.27) were increased in the general population during the early phase of the pandemic compared with prepandemic conditions, mental burden was not increased in patients as well as healthcare workers, irrespective of COVID-19 patient contact. Specific outcome measures (eg, Patient Health Questionnaire) and older comparative data (published ≥5 years ago) were associated with increased mental burden. Across the three population groups, existing mental disorders, female sex, and concerns about getting infected were repeatedly reported as risk factors, while older age, a good economic situation, and education were protective. Conclusions This meta-analysis paints a more differentiated picture of the mental health consequences in pandemic situations than previous reviews. High-quality, representative surveys, high granular longitudinal studies, and more research on protective factors are required to better understand the psychological impacts of the SARS-CoV-2 pandemic and to help design effective preventive measures and interventions that are tailored to the needs of specific population groups.