Esophagectomy for esophageal cancer is invasive thoracic surgery with a high incidence rate of postoperative complications and prolongation of hospitalization, even if the standardized clinical pathway improves the outcome (mortality and morbidity). Postoperative recurrent nerve paralysis (RNP) is related to respiratory complications concomitant with prolonged hospitalization. However, it has not been elucidated which factors affect the incidence and recovery of RNP. To detect the predictive factor for postoperative RNP, we focused on preoperative serum albumin. Patients who had esophageal cancer with standard esophagectomy were evaluated. In total, 94 patients were divided into three groups depending on the presence of RNP (46 in patients without RNP, 29 in those with transient RNP who recovered within 6 months follow-up and 19 in those with residual RNP). We retrospectively investigated factors associated with residual RNP. Preoperative lower serum albumin was associated with residual RNP. In addition, days to the resumption of oral intake and duration of stay in the hospita postoperatively were delayed in the group of residual RNP. Multiple regression analysis indicated that preoperative serum albumin was a predictive factor for residual RNP. Preoperative lower serum albumin level might be linked to residual RNP which could prolong the resumption of postoperative oral intake and shorten the period of stay at the hospital after esophagectomy, leading to unfavorable outcomes for patients.