In the last two years, governments of many countries imposed heavy social restrictions to contain the spread of the COVID-19 virus, with consequent increase of bad mood, distress, or depression for the people involved. Few studies investigated the impact of these restrictive measures on individual social proficiency, and specifically the processing of emotional facial information, leading to mixed results. The present research aimed at investigating systematically whether, and to which extent, social isolation influences the processing of facial expressions. To this end, we manipulated the social exclusion experimentally through the well-known Cyberball game (within-subject factor), and we exploited the occurrence of the lockdown for the Swiss COVID-19 first wave by recruiting participants before and after being restricted at home (grouping factor). We then tested whether either form of social segregation influenced the processing of pain, disgust or neutral expressions, across multiple tasks probing access to different components of affective facial responses (state-specific, shared across states). We found that the lockdown (but not game-induced exclusion) affected negatively the processing of pain-specific information, without influencing other components of the affective facial response related to disgust or broad unpleasantness. In addition, participants recruited after the confinement reported lower scores in both empathy questionnaires and affective assessments of Cyberball co-players. These results suggest that social isolation affected negatively individual sensitivity to other people’s affect and, with specific reference to the processing of facial expressions, the processing of pain-diagnostic information.