Reliability of equine visual lameness classification as a function of expertise, lameness severity and rater confidence

Abstract

Visual equine lameness assessment is often unreliable, yet the full understanding of this issue is missing. Here, we investigate visual lameness assessment using near-realistic, three-dimensional horse animations presenting with 0–60 per cent movement asymmetry. Animations were scored at an equine veterinary seminar by attendees with various expertise levels. Results showed that years of experience and exposure to a low, medium or high case load had no significant effect on correct assessment of lame (P>0.149) or sound horses (P≥0.412), with the exception of a significant effect of case load exposure on forelimb lameness assessment at 60 per cent asymmetry (P=0.014). The correct classification of sound horses as sound was significantly (P<0.001) higher for forelimb (average 72 per cent correct) than for hindlimb lameness assessment (average 28 per cent correct): participants often saw hindlimb lameness where there was none. For subtle lameness, errors often resulted from not noticing forelimb lameness and from classifying the incorrect limb as lame for hindlimb lameness. Diagnostic accuracy was at or below chance level for some metrics. Rater confidence was not associated with performance. Visual gait assessment may overall be unlikely to reliably differentiate between sound and mildly lame horses irrespective of an assessor’s background.

Influence of orthopedic examination on lameness scores and interobserver and intraobserver agreement in dogs with naturally occurring elbow osteoarthritis

“…Our high intraobserver agreement is in line with that in previous studies that had higher within‐observer agreement for clinicians compared with students, interns, or residents . On the other hand, years of experience and exposure to a low, medium, or high case load had no significant effect on correct assessment of lameness in an equine study in which a near‐realistic three‐dimensional animation was used rather than actual clinical cases or video‐graphic assessment . Our results did not change after excluding the scores from the surgical intern, providing evidence that experience had no effect on lameness scores in our study, similarly to the previous equine study .…”

“…On the other hand, years of experience and exposure to a low, medium, or high case load had no significant effect on correct assessment of lameness in an equine study in which a near‐realistic three‐dimensional animation was used rather than actual clinical cases or video‐graphic assessment . Our results did not change after excluding the scores from the surgical intern, providing evidence that experience had no effect on lameness scores in our study, similarly to the previous equine study . However, to the best of our knowledge, a similar study on influence of clinical experience involving assessment of canine lameness by multiple less experienced raters compared with multiple more experienced raters is lacking.…”

Generation of Domains for the Equine Musculoskeletal Rehabilitation Outcome Score: Development by Expert Consensus

“…The highest agreement across the panel was for the inclusion of a lameness measurement within TEMROS. In equine practice lameness is typically evaluated by observing movement asymmetry in trot; however, this often presents a challenge, especially in horses presenting with low grade lameness [41,42]. For gold standard detection and evaluation, force plates are recommended, although these are not used outside the research environment and not practical for clinical assessment.…”

“…Therefore, inertial sensor systems are useful where force plate analysis is not practical [43]. In practice, lameness assessment is commonly conducted by a visual gait assessment without technological equipment [41] and visual assessment, without technological equipment, has been investigated for both intra-and inter-rater reliability. Keegan et al [42] studied the reliability of overground evaluation of lameness to determine if clinicians could agree on whether horses were lame and if so, which was the limb and score for the maximum level of lameness [42].…”

Effect of meloxicam treatment on movement asymmetry in riding horses in training

“…A major shortcoming of subjective visual assessment of lameness is the low inter-observer agreement [4–7], probably in part due to limitations of the human visual perception of asymmetry [8,9]. Several movement symmetry based objective systems have been developed (for a review see Serra Bragança, Rhodin and van Weeren [10]) with the aim of obtaining a more accurate detection of lameness.…”