After long-standing calls for research into “for whom” neighborhood matters, the literature has recently gained traction, focusing on background characteristics such as gender, race, and socioeconomic background. However, there is still scarce and only indirect evidence on whether neighborhood effects vary by children’s academic proneness. In this paper, we combine the Neighborhood Choice Model with quantile regressions to investigate whether neighborhood effects vary by children’s achievement level. Using Norwegian register data, we demonstrate heterogeneity in the effects of neighborhood deprivation as a function of students’ proneness to academic achievement. Students at the very lowest and highest levels of academic achievement are the least affected by neighborhood disadvantage, while those with achievement below average—but not at the lowest end—are most strongly affected. Moreover, while girls are less affected than boys, and children from less educated and affluent families are more affected by neighborhood disadvantage than their more privileged peers, the effects of these observed moderators are considerably smaller than the heterogeneity observed as a function of academic proneness. Overall, our use of quantile regressions opens up new venues for understanding “for whom” neighborhoods matter, with individual-level academic vulnerability and strengths being an influential source of neighborhood effects variation.