Using Surgeon Hand Motions to Predict Surgical Maneuvers

Azari, David P.; Hu, Yu; Miller, Brady L.; Le, Brian; Radwin, Robert G.

doi:10.1177/0018720819838901

Cited by 14 publications

(4 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, integrated eye-tracking can be used for novel gaze-based techniques [ 25 ] to assess gaze, attentiveness and students’ focused engagement with tasks. Integrated hand-tracking may be useful in the assessment of technical skills by allowing entirely new domains of performance and progression to be captured [ 26 , 27 ] through measuring precision and economy of movement that has been shown to be an objective measure of technical skill [ 28 , 29 ].…”

Section: Discussionmentioning

confidence: 99%

The use of mixed reality technology for the objective assessment of clinical skills: a validation study

et al. 2022

View full text Add to dashboard Cite

Background Mixed Reality technology may provide many advantages over traditional teaching methods. Despite its potential, the technology has yet to be used for the formal assessment of clinical competency. This study sought to collect validity evidence and assess the feasibility of using the HoloLens 2 mixed reality headset for the conduct and augmentation of Objective Structured Clinical Examinations (OSCEs). Methods A prospective cohort study was conducted to compare the assessment of undergraduate medical students undertaking OSCEs via HoloLens 2 live (HLL) and recorded (HLR), and gold-standard in-person (IP) methods. An augmented mixed reality scenario was also assessed. Results Thirteen undergraduate participants completed a total of 65 OSCE stations. Overall inter-modality correlation was 0.81 (p = 0.01), 0.98 (p = 0.01) and 0.82 (p = 0.01) for IP vs. HLL, HLL vs. HLR and IP vs. HLR respectively. Skill based correlations for IP vs. HLR were assessed for history taking (0.82, p = 0.01), clinical examination (0.81, p = 0.01), procedural (0.88, p = 0.01) and clinical skills (0.92, p = 0.01), and assessment of a virtual mixed reality patient (0.74, p = 0.01). The HoloLens device was deemed to be usable and practical (Standard Usability Scale (SUS) score = 51.5), and the technology was thought to deliver greater flexibility and convenience, and have the potential to expand and enhance assessment opportunities. Conclusions HoloLens 2 is comparable to traditional in-person examination of undergraduate medical students for both live and recorded assessments, and therefore is a valid and robust method for objectively assessing performance. The technology is in its infancy, and users need to develop confidence in its usability and reliability as an assessment tool. However, the potential to integrate additional functionality including holographic content, automated tracking and data analysis, and to facilitate remote assessment may allow the technology to enhance, expand and standardise examinations across a range of educational contexts.

show abstract

Section: Discussionmentioning

confidence: 99%

The use of mixed reality technology for the objective assessment of clinical skills: a validation study

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Future expansion of digital video records and data mining may alleviate this limitation. Parallel work 41 has automatically classified suturing and tying periods every 2 seconds with 79% accuracy and may be used in the future to further refine measures of fluidity of motion and motion economy within the simulated setting. Future work may also explore how different simulated tissues (eg, foam, balloons, tissue paper) affect assessment scores.…”

Section: Discussionmentioning

confidence: 99%

Modeling Performance of Open Surgical Cases

Azari

Frasier²,

Miller³

et al. 2021

Sim Healthcare

Self Cite

View full text Add to dashboard Cite

Introduction: Previous efforts used digital video to develop computer-generated assessments of surgical hand motion economy and fluidity of motion. This study tests how well previously trained assessment models match expert ratings of suturing and tying video clips recorded in a new operating room (OR) setting Methods: Enabled through computer vision of the hands, this study tests the applicability of assessments born out of benchtop simulations to in vivo suturing and tying tasks recorded in the OR. Results: Compared with expert ratings, computer-generated assessments for fluidity of motion (slope = 0.83, intercept = 1.77, R 2 = 0.55) performed better than motion economy (slope = 0.73, intercept = 2.04, R 2 = 0.49), although 85% of ratings for both models were within ±2 of the expert response. Neither assessment performed as well in the OR as they did on the training data. Assessments were sensitive to changing hand postures, dropped ligatures, and poor tissue contact-features typically missing from training data. Computer-generated assessment of OR tasks was contingent on a clear, consistent view of both surgeon's hands. Conclusions: Computer-generated assessment may help provide formative feedback during deliberate practice, albeit with greater variability in the OR compared with benchtop simulations. Future work will benefit from expanded available bimanual video records.

show abstract

“…Past studies have used HMM to model performance and strategies of experts and novices in the field of healthcare (Azari et al, 2019; Megali et al, 2006; Rosen et al, 2002). The fundamental premise of an HMM is that it describes a system that transitions between states that cannot be directly observed.…”

Section: Methodsmentioning

confidence: 99%

Deriving Effective Decision-Making Strategies of Prosthetists: Using Hidden Markov Modeling and Qualitative Analysis to Compare Experts and Novices

Saravanan

Menold

2021

Hum Factors

View full text Add to dashboard Cite

Objective This research focuses on studying the clinical decision-making strategies of expert and novice prosthetists for different case complexities. Background With an increasing global amputee population, there is an urgent need for improved amputee care. However, current prosthetic prescription standards are based on subjective expertise, making the process challenging for novices, specifically during complex patient cases. Hence, there is a need for studying the decision-making strategies of prosthetists. Method An interactive web-based survey was developed with two case studies of varying complexities. Navigation between survey pages and time spent were recorded for 28 participants including experts ( n = 20) and novices ( n = 8). Using these data, decision-making strategies, or patterns of decisions, during prosthetic prescription were derived using hidden Markov modeling. A qualitative analysis of participants’ rationale regarding decisions was used to add a deep contextualized understanding of decision-making strategies derived from the quantitative analysis. Results Unique decision-making strategies were observed across expert and novice participants. Experts tended to focus on the personal details, activity level, and state of the residual limb prior to prescription, and this strategy was independent of case complexity. Novices tended to change strategies dependent upon case complexity, fixating on certain factors when case complexity was high. Conclusion The decision-making strategies of experts stayed the same across the two cases, whereas the novices exhibited mixed strategies. Application By modeling the decision-making strategies of experts and novices, this study builds a foundation for development of an automated decision-support tool for prosthetic prescription, advancing novice training, and amputee care.

show abstract

Using Surgeon Hand Motions to Predict Surgical Maneuvers

Cited by 14 publications

References 55 publications

The use of mixed reality technology for the objective assessment of clinical skills: a validation study

The use of mixed reality technology for the objective assessment of clinical skills: a validation study

Modeling Performance of Open Surgical Cases

Deriving Effective Decision-Making Strategies of Prosthetists: Using Hidden Markov Modeling and Qualitative Analysis to Compare Experts and Novices

Contact Info

Product

Resources

About