Finding your favorite videos from massive sports video data has become a big demand for users, accurate sports videos can better help people learn sports content, and the traditional data management and retrieval methods using text identifiers are difficult to meet the needs of users, so the research on the extraction of sports objects in sports videos is of great significance. This paper mainly studies and proposes the basketball object extraction method based on image segmentation algorithm and can accurately analyze the trajectory of the basketball target. By modeling the video frame of basketball game, the basketball object is selected for segmentation and extraction. The extracted basketball object can be used for tracking the target in the basketball video clip retrieval system. At the same time, the segmentation and extraction of the basketball object are also the core part in the basketball video clip retrieval framework. Combined with the characteristics of basketball video images in the database, the algorithm extracts the image block variance and contrast to form the training feature vector, and the correct segmentation rate on the database is higher than 95.2%. The results show that this method has a good effect on the segmentation and extraction of basketball objects in basketball videos.