Visualization software provides powerful numerical calculations for the 3D mathematical modeling process. This paper uses optical motion capture mathematical modeling technology to track and detect dancers' dance movements in segments in real-time. The research results found that the visual dance choreography system proposed in the article improves the efficiency of posture analysis and real-time analysis improves the quality of teaching. This has played an important role in the digital teaching of dance movements and the development of science.