Prediction of Voice Fundamental Frequency and Intensity from Surface Electromyographic Signals of the Face and Neck

Abstract

Silent speech interfaces (SSIs) enable speech recognition and synthesis in the absence of an acoustic signal. Yet, the archetypal SSI fails to convey the expressive attributes of prosody such as pitch and loudness, leading to lexical ambiguities. The aim of this study was to determine the efficacy of using surface electromyography (sEMG) as an approach for predicting continuous acoustic estimates of prosody. Ten participants performed a series of vocal tasks including sustained vowels, phrases, and monologues while acoustic data was recorded simultaneously with sEMG activity from muscles of the face and neck. A battery of time-, frequency-, and cepstral-domain features extracted from the sEMG signals were used to train deep regression neural networks to predict fundamental frequency and intensity contours from the acoustic signals. We achieved an average accuracy of 0.01 ST and precision of 0.56 ST for the estimation of fundamental frequency, and an average accuracy of 0.21 dB SPL and precision of 3.25 dB SPL for the estimation of intensity. This work highlights the importance of using sEMG as an alternative means of detecting prosody and shows promise for improving SSIs in future development.

The Characterization of Normal Male and Female Voice from Surface Electromyographic Parameters

“…Electromyographic recording depends on electrode placement and application, skin perspiration and temperature, muscle fatigue, contraction speed, muscle bulk, contamination of nearby muscles, subcutaneous fat thickness, or slight variations in the execution of the task [ 12 , 13 ]. Therefore, the signal must be normalized for interpretation and comparison, which is not simple [ 2 ].…”

“…The dynamic normalization method (with dynamic tasks that were neither isotonic nor isometric) was used in this study, as it has been revealed that there is strong evidence of reduced inter-subject variability compared to other normalization methods [ 13 , 20 ]. The “read-aloud” task was chosen as the reference activity for signal normalization, as it was considered to be one that could provide more information than the other isolated tasks.…”

“…The dynamic normalization method (with dynamic tasks that were neither isotonic nor isometric) was used in this study, as several authors claim that there is strong evidence of reduced inter-subject variability compared to other normalization methods [ 13 ].…”

“…The electromyogram obtained is the result of the sum of the motor unit action potentials produced during the contraction of the neuromuscular units [ 7 , 12 , 13 ]. The amplitude (maximum height of the action potential) is expressed in microvolts, while the frequency is measured in hertz, indicating the number of times the potential is repeated per unit of time (usually in one second) [ 5 ].…”

“…External noise was removed from the SEMG signal with Butterwor type digital filtering. The following data were then obtained (Table1): peak (µV) (m mum muscle activity), the mean of muscle activation (µV), the minimum value of mus activation (µV), and standard deviation (µV) and mean frequency (Hz) of the numbe times the action potential is repeated per unit of time.The dynamic normalization method (with dynamic tasks that were neither isoto nor isometric) was used in this study, as several authors claim that there is strong evide of reduced inter-subject variability compared to other normalization methods[13].The "read-aloud" task was chosen as the baseline activity for signal normalizati as it was felt that it could provide much more information than the other isolated task Electrical activity was also collected during rest or static assessment[4], which w electrically silent and progressively activated in relation to muscle contraction (Table1…”