Quality prediction of synthesized speech based on tensor structured EEG signals

Abstract

This study investigates quality prediction methods for synthesized speech using EEG. Training a predictive model using EEG is challenging due to a small number of training trials, a low signal-to-noise ratio, and a high correlation among independent variables. When a predictive model is trained with a machine learning algorithm, the features extracted from multi-channel EEG signals are usually organized as a vector and their structures are ignored even though they are highly structured signals. This study predicts the subjective rating scores of synthesized speeches, including their overall impression, valence, and arousal, by creating tensor structured features instead of vectorized ones to exploit the structure of the features. We extracted various features to construct a tensor feature that maintained their structure. Vectorized and tensorial features were used to predict the rating scales, and the experimental result showed that prediction with tensorial features achieved the better predictive performance. Among the features, the alpha and beta bands are particularly more effective for predictions than other features, which agrees with previous neurophysiological studies.

Speech Quality Evaluation of Synthesized Japanese Speech Using EEG

“…This study also found which frequency band is useful in order to reduce the complexity of models which will shorten the processing time. In comparison with [9], our study tried to generalize the approach across the subjects while the previous work was done within subject. Therefore, our approach may reduce the prediction performance.…”

“…In [8], they proposed brain computer interface-based equation to predict quality of experience MOS, and achieved 1.00 of root mean squared error (RMSE) between actual and predicted MOS. In addition, by using tensor representation of all channels and all frequency bands, a study conducted by [9] shows that EEG signals could be used to predict MOS, valence, and arousal within the same subject. We also previously examined which EEG electrodes, frequency bands, and time length significantly represent perceived speech quality in Japanese using the generalized fisher scores [10].…”

GMM-Based Evaluation of Synthetic Speech Quality Using 2D Classification in Pleasure-Arousal Scale

“…Apart from the naturalness and understandability of contents, listening tests can also measure the distinguishability of characters or the degree of entertainment [3]. The subjective scales for rating the synthesized speech may include only a few scored parameters, such as an overall impression by a mean opinion score (MOS) describing the perceived speech quality from poor to excellent, a valence from negative to positive, and an arousal from unexcited to excited [4]. The MOS scale can be used not only for naturalness, but for different dimensions, such as affect (from negative to positive) or speaking style (from irritated to calm) as well [5].…”