Surch: Enabling Structural Search and Comparison for Surgical Videos

Surgment: Segmentation-enabled Semantic Search and Creation of Visual Question and Feedback to Support Video-Based Surgery Learning

“…[78] shows that prompting learners to answer reflective questions while watching videos could help them learn. Surch [35] visualizes the steps of surgery videos through graphs and enables viewers to search for videos based on the structure. Prior techniques primarily focus on extracting the procedures in a task from videos, with little focus on helping viewers discern components within one image frame, which is critical in surgery learning.…”

“…Several studies have developed techniques to identify key phases to support video navigation [11,22,23,28,41,42,68,79]. Surch [35] uses procedural graphs to represent surgery videos and enables efficient search of videos from a bank. In Surgment, we build upon prior work and display the keyframes in a surgery recording to help surgeons easily navigate the videos.…”

“…Focusing on user interactions for creating questions and feedback, surgeons also saw potential in the search-by-mask tool for video navigation and learning. Inspired by Surch [35], we envision extending Surgment to allow users to retrieve and compare scenes from multiple surgery videos, further enriching the learning experience.…”

“…However, the surgery video-watching experience has mostly been passive where the viewers have limited opportunities to actively engage with the video content and receive feedback [17,34,83]. Prior work has explored techniques to enhance the interactivity of videos for learning, including supporting viewers to collaboratively create step-by-step descriptions [93] and concept maps [49] for how-to videos, enabling teachers to embed multimedia exercises in lecture videos [36], and presenting the steps in surgery videos for viewers to search based on structures [35].…”