Direct Observation vs. Video-Based Assessment in Flexible Cystoscopy

Dagnæs-Hansen, Julia; Mahmood, Oria; Bube, Sarah; Bjerrum, Flemming; Subhi, Yousif; Rohrsted, Malene; Konge, Lars

doi:10.1016/j.jsurg.2017.10.005

Cited by 19 publications

(14 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They found that direct observation resulted in 10% lower scores for residents and 10% higher scores for consultants when compared to blinded video-based assessments [29] . In a study on assessment of flexible cystoscopy, Dagnaes-Hansen et al [30] also found significant differences between direct and video-based assessments of urologist performance. These findings are in concurrence with our results and emphasise that even in a controlled research set-up and conducted in good faith, direct observation assessments are prone to bias.…”

Section: Discussionmentioning

confidence: 99%

National Implementation of Simulator Training Improves Transurethral Resection of Bladder Tumours in Patients

Bube

Kingo

Madsen

et al. 2022

European Urology Open Science

Self Cite

View full text Add to dashboard Cite

Section: Discussionmentioning

confidence: 99%

National Implementation of Simulator Training Improves Transurethral Resection of Bladder Tumours in Patients

Bube

Kingo

Madsen

et al. 2022

European Urology Open Science

Self Cite

View full text Add to dashboard Cite

“…Current state of the art in robotic assistance [62,63] for surgical procedures with unified master-slave control systems [64,65] has a considerable potential for augmenting the precision and technical capability of physicians, but some challenges still need to be met in terms of optimized system architecture, software, mechanical design, imaging systems, and user interface design and management for maximum safety. To avoid single observer bias [66], objective quantitative performance criteria need to be worked out for defining gold standards of true expert performance in this emerging realm of assistive technology, pushing optimal training programs for novices. Cogently designed and parsimoniously deployed Artificial Intelligence [20,67] can help move things forward in this direction.…”

Section: Discussionmentioning

confidence: 99%

Spatiotemporal Modeling of Grip Forces Captures Proficiency in Manual Robot Control

et al. 2023

View full text Add to dashboard Cite

New technologies for monitoring grip forces during hand and finger movements in non-standard task contexts have provided unprecedented functional insights into somatosensory cognition. Somatosensory cognition is the basis of our ability to manipulate and transform objects of the physical world and to grasp them with the right amount of force. In previous work, the wireless tracking of grip-force signals recorded from biosensors in the palm of the human hand has permitted us to unravel some of the functional synergies that underlie perceptual and motor learning under conditions of non-standard and essentially unreliable sensory input. This paper builds on this previous work and discusses further, functionally motivated, analyses of individual grip-force data in manual robot control. Grip forces were recorded from various loci in the dominant and non-dominant hands of individuals with wearable wireless sensor technology. Statistical analyses bring to the fore skill-specific temporal variations in thousands of grip forces of a complete novice and a highly proficient expert in manual robot control. A brain-inspired neural network model that uses the output metric of a self-organizing pap with unsupervised winner-take-all learning was run on the sensor output from both hands of each user. The neural network metric expresses the difference between an input representation and its model representation at any given moment in time and reliably captures the differences between novice and expert performance in terms of grip-force variability.Functionally motivated spatiotemporal analysis of individual average grip forces, computed for time windows of constant size in the output of a restricted amount of task-relevant sensors in the dominant (preferred) hand, reveal finger-specific synergies reflecting robotic task skill. The analyses lead the way towards grip-force monitoring in real time. This will permit tracking task skill evolution in trainees, or identify individual proficiency levels in human robot-interaction, which represents unprecedented challenges for perceptual and motor adaptation in environmental contexts of high sensory uncertainty. Cross-disciplinary insights from systems neuroscience and cognitive behavioral science, and the predictive modeling of operator skills using parsimonious Artificial Intelligence (AI), will contribute towards improving the outcome of new types of surgery, in particular the single-port approaches such as NOTES (Natural Orifice Transluminal Endoscopic Surgery) and SILS (Single-Incision Laparoscopic Surgery).

show abstract

“…To our knowledge, this study is the first to evaluate the reliability of video-based assessment comprehensively by using both classical test theory and G theory. Nevertheless, several studies have evaluated the reliability of video-based assessment in other surgical/technical procedures 4,11–16 . However, only 3 studies have made a direct comparison of the same raters assessing both physical and video recordings of performances 13,14,16 .…”

Section: Discussionmentioning

confidence: 99%

Are Video Recordings Reliable for Assessing Surgical Performance? A Prospective Reliability Study Using Generalizability Theory

et al. 2022

View full text Add to dashboard Cite

Introduction: Reliability is pivotal in surgical skills assessment. Video-based assessment can be used for objective assessment without physical presence of assessors. However, its reliability for surgical assessments remains largely unexplored. In this study, we evaluated the reliability of video-based versus physical assessments of novices' surgical performances on human cadavers and 3D-printed models-an emerging simulation modality. Methods: Eighteen otorhinolaryngology residents performed 2 to 3 mastoidectomies on a 3D-printed model and 1 procedure on a human cadaver. Performances were rated by 3 experts evaluating the final surgical result using a well-known assessment tool. Performances were rated both hands-on/physically and by video recordings. Interrater reliability and intrarater reliability were explored using κ statistics and the optimal number of raters and performances required in either assessment modality was determined using generalizability theory. Results: Interrater reliability was moderate with a mean κ score of 0.58 (range 0.53-0.62) for video-based assessment and 0.60 (range, 0.55-0.69) for physical assessment. Videobased and physical assessments were equally reliable (G coefficient 0.85 vs. 0.80 for 3D-printed models and 0.86 vs 0.87 for cadaver dissections). The interaction between rater and assessment modality contributed to 8.1% to 9.1% of the estimated variance. For the 3D-printed models, 2 raters evaluating 2 video-recorded performances or 3 raters physically assessing 2 performances yielded sufficient reliability for high-stakes assessment (G coefficient >0.8). Conclusions: Video-based and physical assessments were equally reliable. Some raters were affected by changing from physical to video-based assessment; consequently, assessment should be either physical or video based, not a combination.

show abstract

Direct Observation vs. Video-Based Assessment in Flexible Cystoscopy

Cited by 19 publications

References 29 publications

National Implementation of Simulator Training Improves Transurethral Resection of Bladder Tumours in Patients

National Implementation of Simulator Training Improves Transurethral Resection of Bladder Tumours in Patients

Spatiotemporal Modeling of Grip Forces Captures Proficiency in Manual Robot Control

Are Video Recordings Reliable for Assessing Surgical Performance? A Prospective Reliability Study Using Generalizability Theory

Contact Info

Product

Resources

About