The International GNSS Service analysis centers provide orbit products of GPS satellites with weekly, daily, and sub-daily latency. The most frequent ultra-rapid products, which include one day of orbits derived from observations and one day of orbit prediction, are vital for real-time applications. However, the predicted part of the ultra-rapid orbits is less accurate than the part covered by observations and has deviations of several decimetres with respect to the final products. In this study, we investigate the potential of applying machine learning (ML) and deep learning (DL) algorithms to enhance physics-based orbit predictions further. We employed multiple ML/DL algorithms and comprehensively compared the performances of different models. Since the prediction errors of the physics-based propagators accumulate with time and have sequential characteristics, specific sequential modelling algorithms, such as Long Short-Term Memory (LSTM), show superiority. Our approach shows promising results with average improvements of 47% in 3D RMS within the 24-hour prediction interval of the ultra-rapid products. In the end, we apply the orbits improved by LSTM to kinematic precise point positioning and demonstrate the benefits for geodetic applications. The accuracy of the station coordinates estimated based on these products is improved by 16% on average compared to those using ultra-rapid orbits.