A novel fifth-degree cubature Kalman filter is proposed to improve the accuracy of real-time orbit determination by ground-based radar. A fully symmetric cubature rule, approaching the Gaussian weighted integral of a nonlinear function in general form, is introduced, and the specific points and weights are calculated by matching the monomials of degree not greater than five with the exact values. On the basis of the above rule, a novel fifth-degree cubature Kalman filter, which can achieve a higher accuracy than UKF and CKF, is derived under the Bayesian filtering framework. Then, to describe the nonlinear system more accurately, the orbital dynamics equation with J2 perturbation is used as the state equation, and the nonlinear relationship between the radar measurement elements and orbital states is built as the measurement equation. The simulation results show that, compared with the traditional third-degree algorithm, the proposed fifth-degree algorithm has a higher accuracy of orbit determination.