A Novel Hierarchical Framework for Measuring the Complexity and Irregularity of Multimodal Speech Signals and Its Application in the Assessment of Speech Impairment in Amyotrophic Lateral Sclerosis

Abstract

Purpose The purposes of this study are to develop a novel multimodal framework for measuring variability at the muscular, kinematic, and acoustic levels of the motor speech hierarchy and evaluate the utility of this framework in detecting speech impairment in amyotrophic lateral sclerosis (ALS). Method The myoelectric activities of three bilateral jaw muscle pairs (masseter, anterior temporalis, and anterior belly of digastric), jaw kinematics, and speech acoustics were recorded in 13 individuals with ALS and 10 neurologically healthy controls during sentence reading. Thirteen novel measures (six muscular, three kinematic, four acoustic), which characterized two different but interrelated aspects of variability—complexity and irregularity—were derived using linear and nonlinear methods. Exploratory factor analysis was applied to identify the latent factors underlying these measures. Based on the latent factors, three supervised classifiers—support vector machine (SVM), random forest (RF), and logistic regression (Logit)—were used to differentiate between the speech samples for patients and controls. Results Four interpretable latent factors were identified, representing the complexity of jaw kinematics, the irregularity of jaw antagonists functioning, the irregularity of jaw agonists functioning, and the irregularity of subband acoustic signals, respectively. Based on these latent factors, the speech samples for patients and controls were classified with high accuracy (> 96% for SVM and RF; 88.64% for Logit), outperforming the unimodal measures. Two factors showed significant between-groups differences, as characterized by decreased complexity of jaw kinematics and increased irregularity of jaw antagonists functioning in patients versus controls. Conclusions Decreased complexity of jaw kinematics presumably reflects impaired fine control of jaw movement, while increased irregularity of jaw antagonists functioning could be attributed to reduced synchronization of motor unit firing in ALS. The findings provide preliminary evidence for the utility of the multimodal framework as a novel quantitative assessment tool for detecting speech impairment in ALS and (potentially) in other neuromotor disorders.

Applied Machine Learning Techniques to Diagnose Voice-Affecting Conditions and Disorders: Systematic Literature Review

“…A total of 11/145 (7.6%) studies integrated nonvocal data in conjunction with vocal features to form the input feature sets for the ML models; 6 of these studies [ 54 , 66 , 96 , 105 , 149 , 163 ] incorporated demographic data, including gender, age, BMI, comorbidities, weight, height, and disease duration. Meanwhile, 2 of the studies [ 141 , 153 ] used video inputs, while 3 studies [ 83 , 158 , 159 ] incorporated external sensor signals such as electromyography and motion trackers.…”

A multimodal approach to automated hierarchical assessment of bulbar involvement in amyotrophic lateral sclerosis

“…Moreover, the global reduction of voluntary recruitment, firing rate, and number of viable bulbar motor units, as well as changes in firing rate variability and muscle fiber conduction velocity are expected to modify the amplitude dynamics of the bulbar motor system’s outputs at both higher (e.g., neuromuscular) and lower (e.g., acoustic) levels ( 46 , 49 , 58 ), as encoded across modalities. In addition, reduced voluntary recruitment and number of viable bulbar motor units can also compromise the ability of the bulbar motor system in conveying complex motor commands, which would be manifested by the complexity of the system’s outputs at all levels ( 10 , 45 ). Abnormal discharge patterns of bulbar motor units, as relating to slower and more variable firing rates, more heterogeneous contractile properties, and changes in muscle fiber conduction velocity, can disrupt the rhythms of bulbar motor activities across modalities ( 49 , 58–61 ).…”

“…Abnormal discharge patterns of bulbar motor units, as relating to slower and more variable firing rates, more heterogeneous contractile properties, and changes in muscle fiber conduction velocity, can disrupt the rhythms of bulbar motor activities across modalities ( 49 , 58–61 ). Lastly, increased variability and heterogeneity of the firing rates and contractile properties of bulbar motor units tend to globally reduce the regularity of bulbar motor activities across modalities ( 10 , 45 ). Taken together, these evidenced-based explanatory links comprise a conceptual framework, illustrating the mechanistic relevance of the selected seven constructs to bulbar neuromuscular pathology in ALS.…”

Machine Learning Prediction of Tongue Pressure in Elderly Patients with Head and Neck Tumor: A Cross-Sectional Study

“…Previous studies of factors that affect MTP have employed multiple regression analysis in healthy elderly and younger adults, but rarely in patients with head and neck tumors [7,14,25]. RF has been applied to speech impairment [26], but a lack of studies has investigated the use of machine learning models to predict other oral functions, such as tongue pressure, in elderly patients with head and neck tumors.…”