Non-SARS-CoV-2 respiratory viral infections, such as influenza virus (FluV) and human respiratory syncytial virus (RSV), have contributed considerably to the burden of infectious diseases in the non-COVID-19 era. While the rates of co-infection in SARS-CoV-2-positive group (SCPG) patients have been determined, the burden of other respiratory viruses in the SARS-CoV-2-negative group (SCNG) remains unclear. Here, we conducted a cross-sectional study (São José do Rio Preto county, Brazil), and we collected our data using a meta-analysis to evaluate the pooled prevalence of FluV and RSV among SCNG patients. Out of the 901 patients suspected of COVID-19, our molecular results showed positivity of FluV and RSV in the SCNG was 2% (15/733) and 0.27% (2/733), respectively. Co-infection with SARS-CoV-2 and FluV, or RSV, was identified in 1.7% of the patients (3/168). Following our meta-analysis, 28 studies were selected (n = 114,318 suspected COVID-19 patients), with a pooled prevalence of 4% (95% CI: 3–6) for FluV and 2% (95% CI: 1–3) for RSV among SCNG patients were observed. Interestingly, FluV positivity in the SCNG was four times higher (OR = 4, 95% CI: 3.6–5.4, p < 0.01) than in the SCPG. Similarly, RSV positivity was significantly associated with SCNG patients (OR = 2.9, 95% CI: 2–4, p < 0.01). For subgroup analysis, cold-like symptoms, including fever, cough, sore throat, headache, myalgia, diarrhea, and nausea/vomiting, were positively associated (p < 0.05) with the SCPG. In conclusion, these results show that the pooled prevalence of FluV and RSV were significantly higher in the SCNG than in the SCPG during the early phase of the COVID-19 pandemic.