The COVID-19 pandemic has introduced new challenges for governments and individuals. Unprecedented efforts at reducing virus transmission launched a novel arena for human face recognition in which faces are partially occluded with masks. Previous studies have shown that masks decrease accuracy of face identity and emotion recognition. The current study focuses on the impact of masks on the speed of processing of these and other important social dimensions. Here we provide a systematic assessment of the impact of COVID-19 masks on facial identity, emotion, gender, and age. Four experiments (N = 116) were conducted in which participants categorized faces on a predefined dimension (e.g., emotion). Both speed and accuracy were measured. The results revealed that masks hindered the perception of virtually all tested facial dimensions (i.e., emotion, gender, age, and identity), interfering with normal speed and accuracy of categorization. We also found that the unwarranted effects of masks were not due to holistic processes, because the Face Inversion Effect (FIE) was generally not larger with unmasked compared with masked faces. Moreover, we found that the impact of masks is not automatic and that under some contexts observers can control at least part of their detrimental effects.