Background: Hospitalized cancer patients suffer a high risk of venous thromboembolism (VTE). Guidelines suggested performing a personalized thromboprophylaxis guided by VTE risk assessment tools. Machine learning (ML) has advantages in data processing and model development. The study aimed to develop predictive models using four different ML methods and to compare their predictive performance.Methods: A retrospective case-control study was conducted from October 1, 2021 to February 30, 2022 in Hunan Cancer Hospital. A total of 1,100 hospitalized cancer patients were included. The outcome variable was the occurrence of VTE during hospitalization. Input variables, including patient, tumor, treatment and laboratory indicators characteristics, were trained for four ML models: Logistic Regression, Support Vector Machine, Random Forest, and extreme gradient boosting (XGBoost). Model performance was compared using the area under the receiver operating characteristic curve (AUROC). Features rankings were achieved according to the permutation scores of selected features in the optimal model.Results: A total of 1,100 patients (mean [SD] age, 54.75[11.08] years; 485[44.09%] male) were included in the study. There were 340 patients in the VTE group and 760 patients in the non-VTE group. XGBoost model showed the best predictive performance among four models, with AUROC value of 0.818 (95%CI: 0.762, 0.870). Performance of other three models were lower with the following AUROCs in the testing set: Logistic Regression, 0.757(95%CI: 0.689,0.816); Support Vector Machine, 0.759(95%CI: 0.697,0.818); and Random Forest, 0.743(95%CI: 0.678,0.808). The most five significant features in XGBoost model were D-dimer, diabetes, hypertension, pleural metastasis and hematological malignancies. Conclusion: Four predictive models were developed using ML algorithms. XGBoost model was the optimal predictive model compared to other three ML models (Logistic Regression, Support Vector Machine, and Random Forest). This study indicates that ML may play an important role in VTE risk estimation among hospitalized cancer patients and provide reference for thromboprophylaxis.