Capecitabine-based neoadjuvant chemoradiotherapy (nCRT) is the standard treatment for locally advanced rectal cancer. The objective of this study is to analyze overall survival (OS), disease-free survival (DFS) and prognostic factors of patients with stage II to III rectal cancer treated with nCRT in our institution. Between March 2014 to June 2020, 121 locally advanced rectal cancer patients were retrospectively reviewed and analyzed. All of the enrolled patients were treated with capecitabine-based nCRT (pelvic radiotherapy: 45–50.4 Gy, 1.8 Gy/d plus concomitant capecitabine-based chemotherapy), total mesorectal excision surgery (surgery was carried out 8–12 weeks after the end of CRT), and capecitabine-based adjuvant chemotherapy. We examined the pathological complete response rate, 3-year OS, 3-year DFS and the other prognostic factors. Kaplan–Meier method and Log-rank test were used to estimate and compare survival rate. With a median follow-up of 36 months, 3-year DFS and 3-year OS was 74.4% and 83.2%, respectively. Among the 121 patients, 24 achieved pathological complete remission (19.8%). After multivariate analysis, ypTNM stage (TNM stage after neoadjuvant therapy) was significantly associated with DFS. Positive mesorectal fasciae (MRF) status on magnetic resonance imaging and ypTNM stage were significantly related to OS. CRT with capecitabine based regimen provides high rates of survival and sphincter preservation with acceptable toxicity. YpTNM stage was significantly associated with DFS; magnetic resonance imaging MRF status and ypTNM stage were significant factors for OS after multivariate analysis. Distant metastasis is the dominant mode of treatment failure, and it is crucial to optimize systemic treatment for newly diagnosed patients.