Objective To assess antibody response to inactivated COVID-19 vaccine in patients with immune-mediated diseases (IMD) among hospital workers and people aged 65 and older. Methods In this cross-sectional study, we studied 82 hospital workers with IMD (mean age: 42.2 ± 10.0 years) and 300 (mean age: 41.7 ± 9.9 years) controls. Among + 65 aged population, we studied 22 (mean age: 71.4 ± 4.5 years) patients and 47 controls (mean age: 70.9 ± 4.8 years). All study subjects had a negative history for COVID-19. Sera were obtained after at least 21 days following the second vaccination. Anti-spike IgG antibody titers were measured quantitatively using a commercially available immunoassay method. Results Patients with IMD were significantly less likely to have detectable antibodies than healthy controls both among the hospital workers (92.7% vs 99.7%, p < 0.001) and elderly population (77.3% vs 97.9%, p = 0.011). Among patients with IMD, those using immunosuppressive or immune-modulating drugs (64/75, 85.3%) were significantly less likely to have detectable antibodies compared to those off treatment (29/29, 100%) ( p = 0.029). Additionally, a negative association between age and the antibody titer categories among patients ( r = − 0.352; p < 0.001) and controls ( r = − 0.258; p < 0.001) were demonstrated. Conclusions Among hospital workers, the vast majority of patients with IMD and immunocompetent controls developed a significant humoral response following the administration of the second dose of inactivated COVID-19 vaccine. This was also true for the elderly population, albeit with lower antibody titers. Immunosuppressive use, particularly rituximab significantly reduced antibody titers. Antibody titers were significantly lower among those aged ≥ 60 years both in patient and control populations. Whether these individuals should get a booster dose warrants further studies. Supplementary Information The online version contains supplementary material available at 10.1007/s00296-021-04910-7.