Sapphire is widely used in high-tech fields such as microelectronics and optoelectronics because of its excellent optical, chemical, and mechanical properties. Higher requirements of surface quality and processing efficiency bring more challenges to achieve global planarization of the C-, A-, and R-plane sapphire substrate. Enhancing chemical action is one of the effective methods to improve the effect of sapphire chemical mechanical polishing. In this paper, in order to improve the removal rate, chitosan oligosaccharide (COS) was used as a complexing agent in sapphire slurry. From the results, it was found that higher material removal rate and lower surface roughness (Sq) were obtained for C-, A-, and R-plane sapphire, when the concentration of COS in the slurry was 0.10 wt%, 0.05 wt%, and 0.05 wt%, respectively. Meanwhile, the reaction mechanism and process between COS and sapphire were revealed. From the Ultraviolet-visible and X-ray photoelectron spectroscopy analysis, it can be concluded that there was a complexation reaction between sapphire wafers and COS, and the reaction product was soluble complex Al(OH)4–/COS, which resulted in an increase in removal rate and a decrease in surface roughness. Such study has certain guiding significance for practical production.