Background: Asymmetric fetal growth and male sex are both associated with adverse neonatal outcome. However, less is known about the influence of asymmetric growth and fetal sex within SGA neonates, a group of infants already at increased risk for adverse neonatal outcomes. The aim of the present study was to provide insight into variance in risk factors for SGA in a fetal sex-and growth symmetry-specific way. Methods: For this prospective, multicenter cohort study, data from the Screening for Pregnancy Endpoints (SCOPE) study were used with 5628 nulliparous participants, of which 633 (11.3%) pregnancies were complicated with SGA and 3376 (60.0%) women had uncomplicated pregnancies. Association between risk factors for SGA, SGA subgroups, and uncomplicated pregnancies were assessed with multivariable analyses. Results: Prevalence of asymmetric growth varied from 45.8% of SGA infants to 5.5% of infants with a customized birthweight > 90th percentile (p < 0.001). Significantly more SGA males had asymmetric growth compared to SGA female infants (51.2% vs 40.4%, p = 0.009). Maternal pre-pregnancy diet and BMI < 20 and ≥ 30 were significantly associated with symmetric SGA but not with asymmetric SGA. Asymmetric SGA infants had not only lower customized birthweight percentile (4.4 (SD 2.8) vs 5.0 (SD 3.0), p < 0.001), but also lower rates of stillbirth (p = 0.041) and less often Apgar scores < 7 (p = 0.060). Conclusions: Among SGA infants, low customized birthweight percentiles and male sex are associated with asymmetric growth. Only symmetric SGA is significantly associated with maternal risk factors in early pregnancy. There is a substantial variance in risk factors and neonatal outcomes for SGA based on growth symmetry, implying a different pathogenesis.