Intra‐ and interobserver reliability estimates for identification and grading of upper respiratory tract abnormalities recorded in horses at rest and during overground endoscopy

Abstract

Observers were consistent in grading URT disorders. However, significant disparity in grading existed between observers for some conditions affecting reliability.

Clinical insights: Upper respiratory tract obstruction

“…Arytenoid function was graded using the 7‐point Havemeyer scale and then reclassified into binary grades (grade < 3.1 or grade ≥ 3.1). For experienced clinicians, intraobserver agreement for the ordinal scale was substantial as reported by McGivney et al , and excellent for the binary scale. Interobserver agreement for experienced clinicians was substantial for the ordinal grades differing from McGivney et al who reported only fair agreement.…”

“…For experienced clinicians, intraobserver agreement for the ordinal scale was substantial as reported by McGivney et al , and excellent for the binary scale. Interobserver agreement for experienced clinicians was substantial for the ordinal grades differing from McGivney et al who reported only fair agreement. However, McGivney et al found that agreement was better when using Krippendorph’s α (0.59) since this type of analysis considers the proximity of the different observers’ grades, indicating that some observers consistently graded higher or lower than other observers.…”

“…Specifically, when evaluating URT disease presence and severity in exercising horses, sources of measurement disparity need to be identified and may include variations in observer grading of abnormalities, workload, equipment settings and inherent intra‐horse variation of disease presentation. Evaluation of intra‐ and interobserver agreement for grading of resting and exercising URT endoscopic examinations has been recently described . This study assessed the agreement of four observers of equal clinical experience (>10 years) when blindly and independently evaluating 86 videos of resting and OGE exams using published grading scales.…”

Highlights of recent clinically relevant papers

“…This study by C.L. McGivney and colleagues in Ireland and Norway evaluated observer agreement in upper respiratory tract (URT) endoscopy for multiple URT disorders, during resting and overground endoscopy .…”

Impact of pharyngeal endoscopic tip placement and water flushing interval on upper respiratory tract disorders in horses undergoing overground endoscopy

“…Overground endoscopy (OGE) is now commonly used for diagnosis of upper respiratory tract (URT) disorders in horses during exercise [1][2][3][4][5][6]. However, variability in exercising endoscopy findings have been recognised to occur in association with different exercise intensities, head positions, tack and inter-observer variation in URT disorder grading [7][8][9][10][11][12].…”