A hybrid approach to structural modeling of individualized HRTFs

Miccini, Riccardo; Spagnol, Simone

doi:10.1109/vrw52623.2021.00022

Cited by 9 publications

(10 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The technique employed to individualize the HRTFs evaluated herein is based on a recently proposed structural HRTF model [13]. The model combines deep learning (DL) and conventional DSP sub-systems, and features synthesized, selected, and measured components.…”

Section: Hrtf Individualizationmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of Individualized HRTFs in a 3D Shooter Game

Andersen

Miccini

Serafin

et al. 2021

2021 Immersive and 3D Audio: From Architecture to Automotive (I3DA)

View full text Add to dashboard Cite

Previous research stresses the importance of Head-Related Transfer Function (HRTF) individualization approaches for accurately locating sound sources in virtual 3D spaces. However, in the realm of interactive experiences, methods for assessing whether individualized HRTFs bring a benefit to the player experience are rarely investigated. Methods to improve spatial audio rendering are needed now than ever since Virtual Reality (VR) is becoming a mainstream technology for interactive experiences. This paper proposes a method of using in-game metrics to test the hypothesis that individualized HRTFs improve the experience of both expert and novice players in a First-Person Shooter (FPS) game on a desktop environment. The FPS game provides players with a localization task across three different audio renderings using the same acoustic spaces: stereo panning (control condition), generic binaural rendering, and individualized binaural rendering. Collected metrics from the game include localization error, spatial quality attributes, and an extensive questionnaire. The individualized HRTFs for each participant were synthesized using a hybrid structural model. The model employs a deep learning architecture to synthesize a pinna-related response from a pinna image, and combines it with a measured generic head-and-torso response. The interaural time difference (ITD) is then adjusted to match that of an HRTF dataset subject minimizing a localization error metric. The results show that the 22 participants performed significantly better in the localization task with their individualized HRTF. Increased localization accuracy with respect to the generic HRTF was recorded both in azimuth and elevation perception, and especially in the case of expert game players.

show abstract

Section: Hrtf Individualizationmentioning

confidence: 99%

“…For a more comprehensive explanation of the individualization method, as well as a discussion of its performances as derived objectively using spectral distortion metrics and a sagittal-plane localization model, please see [13].…”

Section: B Hrtf Set Generationmentioning

confidence: 99%

Evaluation of Individualized HRTFs in a 3D Shooter Game

Andersen

Miccini

Serafin

et al. 2021

2021 Immersive and 3D Audio: From Architecture to Automotive (I3DA)

View full text Add to dashboard Cite

show abstract

“…In previous work, Lee and Kim [25] utilized both a pinna image and anthropometric features of the head and torso as inputs to predict individual HRTFs. More recently, based on experimental findings highlighting the role of ear pinna in generating spectral cues of HRTFs [26], DNN architectures that generate the magnitude spectra of individual HRTFs using only pinna images have been proposed [27], [28]. These DNNs typically employ an autoencoder structure, known for efficient dimensionality reduction of HRTFs [29].…”

Section: Introductionmentioning

confidence: 99%

PRTFNet: HRTF Individualization for Accurate Spectral Cues Using a Compact PRTF

Ko,

Lee,

Nam

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Spatial audio rendering relies on accurate localization perception, which requires individual head-related transfer functions (HRTFs). Previous methods based on deep neural networks (DNNs) for predicting HRTF magnitude spectra from pinna images used HRTF log-magnitude as the network output during the training stage. However, HRTFs encompass the acoustical characteristics of the head and torso, making it challenging to reconstruct the spectral cues necessary for elevation localization. To tackle this issue, we propose PRTFNet to reconstruct the individual spectral cues in HRTFs by mitigating the influence of the head and torso. PRTFNet consists of an end-to-end convolutional neural network (CNN) model and leverages a compact pinna-related transfer function (PRTF) that eliminates the impact of sound reflections from the head and torso in the head-related impulse response (HRIR) as network output. Additionally, we introduce HRTF phase personalization, a technique that utilizes the phase spectra of a selected HRTFs from a database and adjusts the phase by multiplying it by the ratio of the target listener's head width to that of the subject of the selected HRTFs. We evaluated the proposed HRTF individualization methods using the HUTUBS dataset, and the results demonstrate that PRTFNet is highly effective in reconstructing the first and second spectral cues. In terms of log spectral distortion (LSD) and effective LSD (LSDE), PRTFNet outperforms previous deep learning-based model. Furthermore, multiplying the selected phase by the head width ratio reduces the root mean square error (RMSE) of interaural time difference (ITD) by 0.003 ms.INDEX TERMS Head-related transfer functions, individualization, pinna-related transfer functions, spectral cues, spatial hearing.

show abstract

“…The principal component analysis (PCA) and regression analysis were combined for estimating HRTFs [10][11] [12][13] [14]. In recent years, with deep learning widely used in almost all research fields, deep neural networks (DNNs) were proposed to estimate HRIRs using anthropometric features [15], and even to generate HRTFs from the image of the pinna [16]. However, DNN-based methods require a large amount of training data to avoid over-fitting.…”

Section: Introductionmentioning

confidence: 99%

Ensemble Learning Based HRTF Personalization Using Anthropometric Features

SHEN¹,

KUO²,

SHIH³

2022

Preprint

View full text Add to dashboard Cite

<p>In this paper, we propose an ensemble learning based model to synthesize the logarithmic magnitude response of head-related transfer function (HRTF) using anthropometric features. We first cluster subjects based on relevant anthropometric features to reduce differences within each group, then we use the ensemble learning algorithm on clustered results to predict the log-magnitude HRTF. In the training phase, three deep neural networks (DNNs), each of which aims to predict log-magnitude HRTFs in a particular group, are trained using anthropometric and angle-related features. Afterward, another DNN is trained to integrate estimates from the three group-wise DNNs into log-magnitude HTRFs. The proposed model is compared with a baseline DNN model and our previously proposed model, which incorporates an auto-encoder for dimensionality reduction. Experimental results show that the proposed model performs the best in synthesizing log-magnitude HRTFs in terms of the log-spectral distortion (LSD) measure with great stability.</p>

show abstract

A hybrid approach to structural modeling of individualized HRTFs

Cited by 9 publications

References 24 publications

Evaluation of Individualized HRTFs in a 3D Shooter Game

Evaluation of Individualized HRTFs in a 3D Shooter Game

PRTFNet: HRTF Individualization for Accurate Spectral Cues Using a Compact PRTF

Ensemble Learning Based HRTF Personalization Using Anthropometric Features

Contact Info

Product

Resources

About