Efficient Sound Event Localization and Detection in the Quaternion Domain

Dual Quaternion Ambisonics Array for Six-Degree-of-Freedom Acoustic Representation

“…In this section, we present the experimental evaluation and the results discussion. To be consistent with previous literature where novel hypercomplex models are usually compared with their realvalued and quaternion-valued counterparts [35], we evaluate the performance of the proposed approach against the real-valued SELD-TCN baseline [32] and its quaternion counterpart, QSELD-TCN [15]. We slightly modify the latter to properly process signals from two Ambisonics, thus we create two parallel Q-Conv-TC blocks in order to involve a quaternion for each microphone, resulting in an architecture similar to the DualQSELD-TCN parallel in Fig.…”

“…Quaternion Neural Networks (QNNs) define each input, weight, bias and output as a quaternion in (1). Therefore, the multiplication in (15) becomes a multiplication between two quaternions and has to be performed following the Hamilton product in (2). The quaternion FC (Q-FC) layer with a weight matrix W = W W + W X î + W Y  + W Z κ is then:…”

“…To this end, we propose a dual quaternion sound event localization and detection network with temporal convolutional blocks (DualQSELD-TCN) inspired by [15]. In this model, we exploit operations in the dual quaternion domain, including the product in (7) for neural layers and we enclose the two microphones signals as in (10).…”

“…More in detail, the first architecture block comprises three DualQ-Conv2D layers, each followed by batch normalization layer, ReLU activation function, max pooling applied to the frequency axis and dropout. The input should then be properly organized to be fed into the DualQ-TC block, therefore we stack frequency and channel axis together [15]. The core part of our model is the TC block, which has been introduced to remove the computational-heavy sequential processing of recurrent networks [32].…”

“…Indeed, QNNs handle the four signals as a single component and thanks to the quaternion algebra properties, in particular the Hamilton product, they preserve signals relations and correlations. These models have shown interestingly results in different tasks including sound event localization and detection (SELD), which is the task of jointly learning the temporal and spatial location of a sound and its class [13][14][15][16][17]. Despite QNNs success in spatial source localization, no tests have been made with multiple Ambisonics microphones since these networks do not properly handle non-4D inputs due to the four-dimensional nature of quaternion numbers.…”