Development of a Machine-Learning-Based Classifier for the Identification of Head and Body Impacts in Elite Level Australian Rules Football Players

Abstract

Background: Exposure to thousands of head and body impacts during a career in contact and collision sports may contribute to current or later life issues related to brain health. Wearable technology enables the measurement of impact exposure. The validation of impact detection is required for accurate exposure monitoring. In this study, we present a method of automatic identification (classification) of head and body impacts using an instrumented mouthguard, video-verified impacts, and machine-learning algorithms.Methods: Time series data were collected via the Nexus A9 mouthguard from 60 elite level men (mean age = 26.33; SD = 3.79) and four women (mean age = 25.50; SD = 5.91) from the Australian Rules Football players from eight clubs, participating in 119 games during the 2020 season. Ground truth data labeling on the captures used in this machine learning study was performed through the analysis of game footage by two expert video reviewers using SportCode and Catapult Vision. The visual labeling process occurred independently of the mouthguard time series data. True positive captures (captures where the reviewer directly observed contact between the mouthguard wearer and another player, the ball, or the ground) were defined as hits. Spectral and convolutional kernel based features were extracted from time series data. Performances of untuned classification algorithms from scikit-learn in addition to XGBoost were assessed to select the best performing baseline method for tuning.Results: Based on performance, XGBoost was selected as the classifier algorithm for tuning. A total of 13,712 video verified captures were collected and used to train and validate the classifier. True positive detection ranged from 94.67% in the Test set to 100% in the hold out set. True negatives ranged from 95.65 to 96.83% in the test and rest sets, respectively.Discussion and conclusion: This study suggests the potential for high performing impact classification models to be used for Australian Rules Football and highlights the importance of frequencies <150 Hz for the identification of these impacts.

Performance of Oral Cavity Sensors: A Systematic Review

“…Mouthguards were found to have an overall sensitivity ranging from 69.2% to 100% and a positive predictive value ranging from 55.0 to 96.4% depending on the device, minimum impact threshold, and data processing technique [ 85 , 86 , 88 , 89 ]. Using a series of untuned classifiers, one study reported an average true positives rate of 77.84% and a true negative rate of 89.55% [ 91 ]. The best classifier identified was an XGBoost-based model with true positive rates ranging from 94.67 to 100% and true negative rates ranging from 95.65 to 96.83% depending on the used dataset [ 91 ].…”

“…Using a series of untuned classifiers, one study reported an average true positives rate of 77.84% and a true negative rate of 89.55% [ 91 ]. The best classifier identified was an XGBoost-based model with true positive rates ranging from 94.67 to 100% and true negative rates ranging from 95.65 to 96.83% depending on the used dataset [ 91 ].…”

A neural network for the detection of soccer headers from wearable sensor data

“…This, once again, illustrates the need to additionally verify or filter sensor recordings by secondary sources of information in order to obtain reliable exposure data 16 , 17 . Since previous studies demonstrated the potential of data-driven machine learning models for the identification of various types of head impacts in both American 26 , 27 and Australian rules football 28 as well as soccer 29 , 30 , we used an LSTM neural network for the automatized detection of header events as the most common type of RHI in the game of soccer 19 , 23 .…”

“…While their algorithm was able to correctly classify 88% of all sensor-recorded events, only half (51%) of the predicted impacts actually corresponded to actual head impacts (precision) 25 . In actual sporting contexts, most approaches focused on the discrimination between true head impacts and non-impacts in American 26 , 27 or Australian rules football 28 . While the models employed by Wu et al 26 (support vector machine, SVM) and Gabler et al 27 (Adaboost) achieved classification performances ranging from 68.5% to 93.8% and 81.6% to 98.3%, respectively, the best-performing classifier (XGBoost) in the study of Goodin et al 28 showed promising impact and non-impact detection rates of approx.…”

Tacklers’ Head Inertial Accelerations Can Be Decreased by Altering the Way They Engage in Contact with Ball Carriers’ Torsos

“…Recent attempts have been made to accurately quantify and monitor a player's in-game exposure to direct and indirect head impacts in collisions sports via accelerometry (50,51). These devices use linear acceleration magnitude threshold of >5g (52) or ≥10g (50,51,53) to identify an impact event. These thresholds are well above this current study's average peak linear acceleration magnitude reported but are not too dissimilar to a recent in-game study of professional rugby league players tackling (linear (median, 7.1g; Q1, 5.2g; Q3, 9.9g) and angular acceleration (median, 0.6 krad•s −2 ; Q1, 0.4 krad•s 2−1 ; Q3, 0.9 krad•s −2 )).…”