Worldwide, South Korea had the second highest incidence rates of colorectal cancer (CRC) in 2018. To inform public health policy to prevent CRC, we aimed to identify major modifiable lifestyle factors underlying the alarming increase in CRC incidence. We obtained information on CRC statistics from the Korea National Cancer Incidence Database and on the distribution of dietary and lifestyle factors known to modify CRC risk from the Korea National Health and Nutrition Examination Survey. To examine time trends between 2001 and 2013, we calculated annual percent changes of CRC incidence rates and of prevalence of etiologic factors by sex and age. Across all sex and age groups, the most commonly diagnosed cancer was rectal cancer while the most rapidly increasing cancer was distal colon cancer. For the lifestyle factors examined, decreases in exercise were observed across all age groups of both sexes. Yet, obesity and alcoholic drinks appear more relevant CRC contributor to men, smoking to women aged 30–49 years, and processed meat intake to adults aged 30–49 years. The heterogeneous results suggest that dietary and lifestyle target to prevent CRC be tailored by sex and age.