In parallel with the rise in interest and volume of social cognition research, there has been increasing awareness of a lack of agreement on the concepts and taxonomy used to study social processes. Two central concepts in the field, Empathy and Theory of Mind (ToM), have been applied as overlapping umbrella terms for a variety of different processes of limited convergence. Here, we review and integrate evidence of brain activation, brain organization, and behavior into a coherent model of social cognitive processes. We start with a meta-analytic clustering of neuroimaging data across different social cognitive tasks. Results show that understanding others' mental states can be described by a multilevel model of hierarchical structure, similar to models in intelligence and personality research. A higher level describes more broad and abstract classes of functioning, whereas a lower level explicates how functions are applied to concrete contexts, given by specific stimulus and task formats. Specifically, the higher level of our model suggests three groups of neurocognitive processes: (i) Predominantly cognitive processes that are engaged when mentalizing requires selfgenerated cognition decoupled from the physical world. (ii) More affective processes which are engaged when we witness emotions in others, based on shared emotional, motor, and somatosensory representations. (iii) Combined processes which engage cognitive and affective functions in parallel. We discuss how these processes are explained by an underlying principal gradient of structural brain organization. Finally, we validate the model by a review of Empathy and ToM task interrelations found in behavioral studies.