This study aims to compare the prevalence of hypogonadism between male patients with early-onset type 2 diabetes mellitus (T2DM) and late-onset type 2 diabetes. A total of 122 male patients with early-onset T2DM (diagnosis age ≤40 years) and 100 male patients with late-onset T2DM (diagnosis age >40 years) were recruited from our in-patient department between 1 January 2013 and 28 December 2015. Serum FSH, LH, testosterone, lipid profile, uric acid, HbA1c, and beta-cell function were determined in blood samples. The diagnosis of hypogonadism was based on the levels of LH, FSH, and total testosterone. The mean onset age was 29.86 ± 6.31 and 54.47 ± 9.97 years old in the early-onset group and late-onset group, respectively. Compared with late-onset T2DM, those with early-onset T2DM had a higher proportion of new-onset diabetes, were more likely to be obese, and had worse glycemic control, lipid control, and lower sex hormone-binding globulin (SHBG). The prevalence of hypogonadism was much higher in the early-onset group than in the late-onset group (48.0% vs. 26.7%, p< 0.05). The rate of secondary hypogonadism in the early-onset group and late-onset group were 44.3% and 25.0%, respectively (p < 0.05). Obesity, waist circumference, and SHBG were significantly associated with serum total testosterone level in all, early-onset, and late-onset T2DM. Both all and early-onset T2DM groups had positive correlations between total testosterone and fasting C-peptide, total cholesterol, triglycerides, and uric acid. Our results indicate that in a population of admission to a large urban hospital in China, the prevalence of hypogonadism was higher in the patients with early-onset T2DM than that of late-onset T2DM. This prevalence might be attributable to greater obesity, worse lipid control, and lower SHBG levels in those patients.