Background: Prostate cancer patients are known to suffer from poor sexual and urinary long-term side-effects following treatment, potentially impacting quality of life. The purpose of our study was to compare health-related quality of life at 3 years between prostate cancer patients and healthy controls according to key lifestyle characteristics. Secondary objectives were to compare urological dysfunction, sexual function, anxiety and depression. Methods: Multicentric, case-control, observational prospective, open, follow-up study including 819 prostate cancer patients < 75 years old from the EPICAP cohort, newly diagnosed from 1 December 2011 to 31 March 2014 and 879 healthy controls. Participants were excluded if they experienced a relapse. Controls from the same geographical region were age-matched and were excluded if they were diagnosed with prostate cancer. Patients received one of the following treatments: active surveillance (AS), radical prostatectomy (RP), external beam radiotherapy (EBRT), High-intensity Focused Ultrasound (HIFU), chemotherapy (CT), or androgen deprivation therapy (ADT) as appropriate. The primary outcome was the quality of life as evaluated by the QLQ-C30 questionnaire. Scores were analyzed by multivariate analysis to adjust for predefined socio-demographic confounding effects. Results: In total, 564 participants were included (mean age 67.9 years): 376 patients and 188 controls. Treatment breakdown was: 258 underwent RP, 90 received EBRT, 52 brachytherapy or HIFU, 15 CT, 26 ADT and 61 AS. There was no difference in median global quality of life between patients and controls (94.87 vs 94.15, p = 0.71). Multivariate analysis showed poorer social functioning in patients (24.3% vs. 16.3%, p = 0.0209), more dyspnea (22% vs. 12.4%, p = 0.0078), and yet less current pain (23% vs 33%, p = 0.0151).