Utility of the Simulated Outcomes Following Carotid Artery Laceration Video Data Set for Machine Learning Applications

Kugener, Guillaume; Pangal, Dhiraj J.; Cardinal, Tyler; Collet, Casey; Lechtholz-Zey, Elizabeth; Lasky, Sasha; Sundaram, Shivani; Markarian, Nicholas; Zhu, Yichao; Roshannai, Arman; Sinha, Aditya; Han, Xuehui; Papyan, Vardan; Hung, Andrew J.; Anandkumar, Animashree; Wrobel, Bozena; Zada, Gabriel; Donoho, Daniel A.

doi:10.1001/jamanetworkopen.2022.3177

Cited by 9 publications

(25 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Full details of the experimental setup, data collection, consent, and implementation parameters for the SOCAL data set can be found in the associated publication. 24,26 Briefly, 75 surgeons ranging from junior surgical trainees to world experts on endoscopic endonasal approaches were recorded in a training exercise for internal carotid artery injury (ICAI). 24,26 The SOCAL surgical video database was developed based on intraoperative video taken from these simulation trials.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Pilot Analysis of Surgeon Instrument Utilization Signatures Based on Shannon Entropy and Deep Learning for Surgeon Performance Assessment in a Cadaveric Carotid Artery Injury Control Simulation

Balu,

Pangal,

Kugener

et al. 2023

Operative Neurosurgery

Self Cite

View full text Add to dashboard Cite

BACKGROUND AND OBJECTIVES: Assessment and feedback are critical to surgical education, but direct observational feedback by experts is rarely provided because of time constraints and is typically only qualitative. Automated, video-based, quantitative feedback on surgical performance could address this gap, improving surgical training. The authors aim to demonstrate the ability of Shannon entropy (ShEn), an information theory metric that quantifies series diversity, to predict surgical performance using instrument detections generated through deep learning. METHODS: Annotated images from a publicly available video data set of surgeons managing endoscopic endonasal carotid artery lacerations in a perfused cadaveric simulator were collected. A deep learning model was implemented to detect surgical instruments across video frames. ShEn score for the instrument sequence was calculated from each surgical trial. Logistic regression using ShEn was used to predict hemorrhage control success. RESULTS: ShEn scores and instrument usage patterns differed between successful and unsuccessful trials (ShEn: 0.452 vs 0.370, P < .001). Unsuccessful hemorrhage control trials displayed lower entropy and less varied instrument use patterns. By contrast, successful trials demonstrated higher entropy with more diverse instrument usage and consistent progression in instrument utilization. A logistic regression model using ShEn scores (78% accuracy and 97% average precision) was at least as accurate as surgeons' attending/resident status and years of experience for predicting trial success and had similar accuracy as expert human observers. CONCLUSION: ShEn score offers a summative signal about surgeon performance and predicted success at controlling carotid hemorrhage in a simulated cadaveric setting. Future efforts to generalize ShEn to additional surgical scenarios can further validate this metric.

show abstract

Section: Methodsmentioning

confidence: 99%

“…In this study, we evaluate the ability of ShEn and DL to predict surgical outcomes from a validated and high-fidelity cadaveric simulation of carotid artery lacerations. 11,14,[24][25][26] This work demonstrates a novel surgical APM to quantify patterns in instrument usage, and we display its performance using DL instrument detections.…”

mentioning

confidence: 99%

Pilot Analysis of Surgeon Instrument Utilization Signatures Based on Shannon Entropy and Deep Learning for Surgeon Performance Assessment in a Cadaveric Carotid Artery Injury Control Simulation

Balu,

Pangal,

Kugener

et al. 2023

Operative Neurosurgery

Self Cite

View full text Add to dashboard Cite

show abstract

“…This collection of annotated videos and corresponding performance data is termed the Simulated Outcomes Following Carotid Artery Laceration (SOCAL) video data set and is openly available in figshare at https:// doi.org/10.6084/m9.figshare.15132468.v1. 17…”

Section: Simulated Outcomes Following Carotid Artery Laceration Data ...mentioning

confidence: 99%

Deep Neural Networks Can Accurately Detect Blood Loss and Hemorrhage Control Task Success From Video

et al. 2022

Self Cite

View full text Add to dashboard Cite

BACKGROUND: Deep neural networks (DNNs) have not been proven to detect blood loss (BL) or predict surgeon performance from video. OBJECTIVE: To train a DNN using video from cadaveric training exercises of surgeons controlling simulated internal carotid hemorrhage to predict clinically relevant outcomes. METHODS: Video was input as a series of images; deep learning networks were developed, which predicted BL and task success from images alone (automated model) and images plus human-labeled instrument annotations (semiautomated model). These models were compared against 2 reference models, which used average BL across all trials as its prediction (control 1) and a linear regression with time to hemostasis (a metric with known association with BL) as input (control 2). The root-mean-square error (RMSE) and correlation coefficients were used to compare the models; lower RMSE indicates superior performance. RESULTS: One hundred forty-three trials were used (123 for training and 20 for testing). Deep learning models outperformed controls (control 1: RMSE 489 mL, control 2: RMSE 431 mL, R 2 = 0.35) at BL prediction. The automated model predicted BL with an RMSE of 358 mL (R2 = 0.4) and correctly classified outcome in 85% of trials. The RMSE and classification performance of the semiautomated model improved to 260 mL and 90%, respectively. CONCLUSION: BL and task outcome classification are important components of an automated assessment of surgical performance. DNNs can predict BL and outcome of hemorrhage control from video alone; their performance is improved with surgical instrument presence data. The generalizability of DNNs trained on hemorrhage control tasks should be investigated.

show abstract

“…With the continuous emergence of various media and short videos [ 1 , 2 ] in recent years, the impact on children's emotions [ 3 , 4 ] in daily life is getting bigger and bigger, such as popular music and videos published on YouTube [ 5 , 6 ], TikTok [ 7 , 8 ], and other platforms. Often these data contain three types of modalities, namely, video [ 9 , 10 ], audio [ 11 , 12 ], and text information [ 13 ]. Usually different types of music genres [ 14 , 15 ] often use these three types of modalities to convey their own emotions and values, and children belong to a purely unconscious state and are easily affected by these data.…”

Section: Introductionmentioning

confidence: 99%

A Multi-Modal Convolutional Neural Network Model for Intelligent Analysis of the Influence of Music Genres on Children’s Emotions

Qian

Chen

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

The influence of music genres on children’s emotional intelligence is one of the hot topics in the field of multi-modal emotion research. How to fuse multi-modal information has an important impact on children’s emotional analysis. Most of the current research is based on transformer, in which the self-attention mechanism module is improved to achieve the fusion effect of multi-modal information. However, it is difficult for these methods to effectively capture the effective information of different modalities. Therefore, for the task of the influence of music genres on children’s emotions, this paper proposes a transformer-based multi-modal convolutional neural network. The first is to use the BiLSTM sub-network model to extract the video and audio features and use the BERT sub-network to extract the text features. Secondly, this paper uses the improved transformer cross-modal fusion module to effectively fuse different types of modal information. Finally, the transformer module is used to judge the information of different modalities and analyze the emotion from the multi-modal information. At the same time, a large number of experiments prove that the model based on multi-modal convolutional neural network proposed in this paper surpasses other methods in prediction accuracy and effectively improves the accuracy of sentiment classification tasks.

show abstract

Utility of the Simulated Outcomes Following Carotid Artery Laceration Video Data Set for Machine Learning Applications

Cited by 9 publications

References 56 publications

Pilot Analysis of Surgeon Instrument Utilization Signatures Based on Shannon Entropy and Deep Learning for Surgeon Performance Assessment in a Cadaveric Carotid Artery Injury Control Simulation

Pilot Analysis of Surgeon Instrument Utilization Signatures Based on Shannon Entropy and Deep Learning for Surgeon Performance Assessment in a Cadaveric Carotid Artery Injury Control Simulation

Deep Neural Networks Can Accurately Detect Blood Loss and Hemorrhage Control Task Success From Video

A Multi-Modal Convolutional Neural Network Model for Intelligent Analysis of the Influence of Music Genres on Children’s Emotions

Contact Info

Product

Resources

About