Individuals with psychopathy or autism spectrum disorder (ASD) can behave in ways that suggest lack of empathy towards others. However, many different cognitive and affective processes may lead to unempathic behavior and the social processing profiles of individuals with high psychopathic vs. ASD traits are likely different. Whilst psychopathy appears characterized by problems with resonating with others’ emotions, ASD appears characterized by problems with cognitive perspective-taking. In addition, alexithymia has previously been associated with both disorders, but the contribution of alexithymia needs further exploration. In a community sample (N = 110) we show for the first time that although affective resonance and cognitive perspective-taking are related, high psychopathic traits relate to problems with resonating with others’ emotions, but not cognitive perspective taking. Conversely, high ASD traits relate to problems with cognitive perspective-taking but not resonating with others’ emotions. Alexithymia was associated with problems with affective resonance independently of psychopathic traits, suggesting that different component processes (reduced tendency to feel what others feel and reduced ability to identify and describe feelings) comprise affective resonance. Alexithymia was not associated with the reduced cognitive perspective-taking in high ASD traits. Our data suggest that (1) elevated psychopathic and ASD traits are characterized by difficulties in different social information processing domains and (2) reduced affective resonance in individuals with elevated psychopathic traits and the reduced cognitive perspective taking in individuals with elevated ASD traits are not explained by co-occurring alexithymia. (3) Alexithymia is independently associated with reduced affective resonance. Consequently, our data point to different component processes within the construct of empathy that are suggestive of partially separable cognitive and neural systems.