Integrating Audio Signal Processing and Deep Learning Algorithms for Gait Pattern Classification in Brazilian Gaited Horses

Abstract

This study focused on assessing the usefulness of using audio signal processing in the gaited horse industry. A total of 196 short-time audio files (4 s) were collected from video recordings of Brazilian gaited horses. These files were converted into waveform signals (196 samples by 80,000 columns) and divided into training (N = 164) and validation (N = 32) datasets. Twelve single-valued audio features were initially extracted to summarize the training data according to the gait patterns (Marcha Batida—MB and Marcha Picada—MP). After preliminary analyses, high-dimensional arrays of the Mel Frequency Cepstral Coefficients (MFCC), Onset Strength (OS), and Tempogram (TEMP) were extracted and used as input information in the classification algorithms. A principal component analysis (PCA) was performed using the 12 single-valued features set and each audio-feature dataset—AFD (MFCC, OS, and TEMP) for prior data visualization. Machine learning (random forest, RF; support vector machine, SVM) and deep learning (multilayer perceptron neural networks, MLP; convolution neural networks, CNN) algorithms were used to classify the gait types. A five-fold cross-validation scheme with 10 repetitions was employed for assessing the models' predictive performance. The classification performance across models and AFD was also validated with independent observations. The models and AFD were compared based on the classification accuracy (ACC), specificity (SPEC), sensitivity (SEN), and area under the curve (AUC). In the logistic regression analysis, five out of the 12 audio features extracted were significant (p < 0.05) between the gait types. ACC averages ranged from 0.806 to 0.932 for MFCC, from 0.758 to 0.948 for OS and, from 0.936 to 0.968 for TEMP. Overall, the TEMP dataset provided the best classification accuracies for all models. The most suitable method for audio-based horse gait pattern classification was CNN. Both cross and independent validation schemes confirmed that high values of ACC, SPEC, SEN, and AUC are expected for yet-to-be-observed labels, except for MFCC-based models, in which clear overfitting was observed. Using audio-generated data for describing gait phenotypes in Brazilian horses is a promising approach, as the two gait patterns were correctly distinguished. The highest classification performance was achieved by combining CNN and the rhythmic-descriptive AFD.

Classification of crispness of food materials by deep neural networks

“…It was impractical to standardize the biting time since the time range of the sound differed between the different types of food. In conclusion, the crushing of crispy foods revealed noise audio patterns that are opposite to the rhythmic patterns observed in previous work related to the audio of trot of horses (Alves et al, 2021).…”

Classification of Human Emotions Based on Javanese Speech Using Convolutional Neural Network and Multilayer Perceptron

“…MFCC offers a brief representation of the assessed votes for ensuing investigation. The Mel scale is derived from a nonlinear frequency scale transformation designed to closely approximate the human perceptual ability to detect small changes in pitch at both low and high frequencies [20]. The stage process of MFCC can be seen in Figure 2.…”

Classifying horse activities with big data using machine learning

“…Horse activity is a sensitive indicator that may be potentially used to track horse health (Mao et al, 2021;Eerdekens et al, 2021). Monitoring the activity of horses (i.e., walking, eating) can provide useful information about horses' welfare, reproduction, survival, and interaction with humans and other animals (Nunes et al, 2021;Alves et al, 2021). To achieve this, animal activity recognition systems with the aid of wearable sensors and the use of machine learning methods over the gathered data have been developed in the past decade (Casella et al, 2020;Eerdekens et al, 2020;Braganca et al, 2020).…”

“…Until now, various machine learning methods have been utilized for HoAR such as Naive Bayes (NB) (Mao et al, 2021;Kamminga et al, 2019a;Lee et al, 2016), Support Vector Machine (SVM) (Mao et al, 2021;Alves et al, 2021;Braganca et al, 2020;Lee et al, 2018Lee et al, , 2016, Decision Tree (DT) (Mao et al, 2021;Casella et al, 2020;Braganca et al, 2020;Lee et al, 2018), Random Forest (RF) (Eerdekens et al, 2021;Alves et al, 2021;Braganca et al, 2020), K-Nearest Neighbors (KNN) (Casella et al, 2020;Lee et al, 2018), Neural Network (NN) (Alves et al, 2021;Casella et al, 2020;Braganca et al, 2020;Gutierrez-Galan et al, 2018;Lee et al, 2016), Quadratic Discriminant Analysis (QDA) (Braganca et al, 2020), Linear Discriminant Analysis (LDA) (Braganca et al, 2020;Lee et al, 2018), and Extreme Learning Machine (ELM) (Lee et al, 2018). Furthermore, some deep learning methods have been investigated for HoAR such as Convolutional Neural Network (CNN) (Mao et al, 2021;Eerdekens et al, 2021;Alves et al, 2021;Eerdekens et al, 2020), Long-Short Term Memory (LSTM (Nunes et al, 2021;Braganca et al, 2020), Auto-Encoder (AE) (Lee et al, 2018), and Recurrent Neural Network (RNN) (Nunes et al, 2021).…”