While the COIVD-19 has triggered the rapid and large-scale promotion of online courses, it has also exposed the problem of poor user experience on online course platforms. Therefore, this research hopes to propose a complete user experience evaluation system to clarify the deficiencies of the platform and give directions for improvement. This research combines qualitative and quantitative, from the perspective of user experience, integrates data obtained through literature research, interview and network research to determine evaluation indices. The Delphi method and analytic hierarchy process are used to determine the weights of various indices and set the scoring level. It is obtained that the "ease of operation" has the most important impact on the user experience of the online live course platform. Innovatively adopting the t-test method to establish a comparative analysis method for the model application, a standardized process of model application is designed, and a complete evaluation system is constructed. Two pieces of software are selected as experimental samples to verify the model from both statistics and practical applications. The test results show that although the two experimental sample software have significant differences in the scores of multiple second-level indices, the overall scores are still similar, which is consistent with the direct evaluation results. Therefore, the user experience evaluation system proposed in this research is reliable, which can provide a reference for the construction of online live course platform and the optimization of user experience. INDEX TERMS User experience, evaluation index, analytic hierarchy process, online live course platform.