Empathy is at the core of our social world, yet multidomain factors that affect its development in socially sensitive periods such as adolescence, are incompletely understood. To address this gap, this study investigated associations between social environmental and mental health factors, and their temporal changes, on adolescent empathetic behaviors (and, for comparison, callous unemotional (CU) traits), in the early longitudinal Adolescent Brain Cognitive Development sample (baseline: n = 11062; 2-year follow-up: n = 9832, median age = 119 and 144 months, respectively). Showing affection towards the youth, liking school, having a close friend and importance of religious beliefs/spirituality in the youth’s life were consistently positively correlated with empathetic behaviors/emotions across assessments (p<0.001, Cohen’s f =0.06-0.10). Positive family dynamics and cohesion, living in a neighborhood that shared the family’s values, but also parent history of substance use and (aggregated) internalizing problems were additionally positively associated with one or more empathetic behaviors at follow-up (p<0.001, f =0.06-0.09). In contrast, externalizing problems, anxiety, depression, fear of social situations and being withdrawn were negatively associated with empathetic behaviors and positively associated with CU traits (p<0.001, f =0.06-0.44). The latter were also correlated with being cyberbullied and/or discriminated against, anhedonia and impulsivity, and their interactions with externalizing and internalizing issues. Significant positive temporal correlations of behaviors at the two assessments indicated positive (early) developmental empathetic behavior trajectories, and negative CU traits’ trajectories. Negative changes in mental health adversely moderated positive trajectories and facilitated negative ones. These findings highlight that adolescent empathetic behaviors/emotions are positively affected by multidomain protective social environmental factors, but are also highly vulnerable to negative factors in the same domains, as well as bully victimization, discrimination and mental health problems, which instead facilitate the development of CU traits.