Based on the phase space generating functional of Green's function for a system with singular higher-order Lagrangian, we have derived the canonical Ward identities (CWI) for such a system. We give a preliminary application of CWI to Yang-Mills theory with higher-order derivatives, and deduce a new form of gauge ghost proper vertices which is different from the other one arising from the BRS invariance.