Exploring the Agreement Between Perceived Rankings and Outcome Data Rankings for Professional Athletic Training Programs

Abstract

Context The U.S. News & World Report college ranking system is used to describe the best graduate programs in the country. Rankings of graduate programs are based solely on perceived ratings of quality by directors and/or deans. Athletic training is not listed by U.S. News & World Report; however, the Commission on Accreditation of Athletic Training Education (CAATE) reports key metrics such as the Board of Certification pass rate and program graduation rate, which could be helpful to create rankings. Objective To evaluate and rank CAATE-accredited professional athletic training (PAT) programs using 2 models: (1) perceived rating of academic quality by program directors (PDs) and (2) CAATE outcome data. Design Cross-sectional with survey and retrospective data. Setting Web-based survey. Patients or Other Participants One hundred fifty-five PDs and 230 CAATE-accredited PAT programs. Main Outcome Measure(s) The perceived rating survey for the PDs resembled the U.S. News & World Report system using a 5-point Likert scale to assess the academic quality of each program. For the CAATE outcome data, we used publicly available information for each PAT program on the CAATE website. We ranked all PAT programs using the data from each model. A Cohen κ was performed to explore the agreement between the PDs’ perceived ranking and the CAATE outcome data rankings. Results No agreement was found between the perceived peer assessment and CAATE outcome data rankings (κ = −0.003, P = .401). Conclusion Perception by PDs did not align with objective data reported by CAATE. The lack of agreement between the 2 ranking systems highlights concerns about using the U.S. News & World Report system for graduate health programs. We suggest exploring a more robust and comprehensive formula including overall pass rate and graduation rate to identify top-ranked programs in athletic training.