Speech recognition with quaternion neural networks

Abstract

Neural network architectures are at the core of powerful automatic speech recognition systems (ASR). However, while recent researches focus on novel model architectures, the acoustic input features remain almost unchanged. Traditional ASR systems rely on multidimensional acoustic features such as the Mel filter bank energies alongside with the first, and second order derivatives to characterize time-frames that compose the signal sequence. Considering that these components describe three different views of the same element, neural networks have to learn both the internal relations that exist within these features, and external or global dependencies that exist between the time-frames. Quaternion-valued neural networks (QNN), recently received an important interest from researchers to process and learn such relations in multidimensional spaces. Indeed, quaternion numbers and QNNs have shown their efficiency to process multidimensional inputs as entities, to encode internal dependencies, and to solve many tasks with up to four times less learning parameters than real-valued models. We propose to investigate modern quaternion-valued models such as convolutional and recurrent quaternion neural networks in the context of speech recognition with the TIMIT dataset. The experiments show that QNNs always outperform real-valued equivalent models with way less free parameters, leading to a more efficient, compact, and expressive representation of the relevant information.32nd Conference on Neural Information Processing Systems (NIPS 2018),

Neural Networks at a Fraction with Pruned Quaternions

“…They found that Q achieved faster convergence on the training loss as well as higher classification accuracy on the test set compared to R. Gaudet and Maida [10] made a similar comparison with image classification on the CIFAR-10 and CIFAR-100 datasets and image segmentation on the KITTI Road Segmentation dataset [9], but this time with Q having a quarter of the number of parameters as R. They reported that on both tasks, quaternion models gave higher accuracy than real and complex networks while having a lower parameter count. Similar advantages for quaternion neural networks over real networks were also found by Parcollet et al [29] for speech recognition.…”

“…Using higher-dimensional data embeddings, such as complex numbers or quaternions, has been successfully shown to reduce model parameters while maintaining accuracy [39,38,10,27]. Quaternions are a 4-dimensional extension to the complex numbers introduced by the mathematician William Rowan Hamilton in 1843 [27], and quaternion neural networks have been built for a variety of ML tasks [43,29,10,28,32,4]. Converting a real model to quaternion can lead to a 75% reduction in model parameters (which is explained in more detail in Sec.…”

A Quaternion Gated Recurrent Unit Neural Network for Sensor Fusion

“…Tay et al explored the use of quaternion networks for lightweight and efficient neural natural language processing in [43]. Parcollet et al investigated the use of quaternion-valued convolutional and recurrent neural networks on speech recognition in [44]. Parcollet et al studied the use of quaternion neural networks for theme identification of telephone conversations in [45].…”

Quaternion-based machine learning on topological quantum systems

“…Specifically, a quaternion number, containing one real part and three imaginary parts, and the corresponding quaternion-based neural networks [39][40][41][42] are expected to enhance the performance on processing of data with more degrees of freedom than the conventional real-number and complex-number systems. There have been various proposals about quaternion-based neural networks in ML techniques and applications in computer science, such as the quaternion convolutional neural network (qCNN) [38,43,44], quaternion recurrent neural network [45], quaternion generative adversarial networks [46], quaternion-valued variational autoencoder [47], quaternion graph neural networks [48], quaternion capsule networks [49] and quaternion neural networks for the speech recognitions [50]. However, the ML-related applications of the quaternion-based neural networks on solving problems in physics are still limited, especially in the topological phase detections, even though the quaternion-related concepts have been applied in some fields in physics [51][52][53].…”