On the Predictability of Hrtfs from Ear Shapes Using Deep Networks

Zhou, Yaxuan; Jiang, Hao; Ithapu, Vamsi Krishna

doi:10.1109/icassp39728.2021.9414042

Cited by 10 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The first architecture used is a modified version of CNN-Reg [2], representing a CNN with skip connections. This architecture features four consecutive stages and was originally applied to predict the single-frequency gammatone-filtered HRTF log-amplitude spectrum jointly for 360 directions from voxelised ear meshes.…”

Section: Network Architecturesmentioning

confidence: 99%

“…Personalised audio is more than ever in demand both by industrial and academic research to achieve a high level of plausibility when binaurally reproducing virtual acoustic environments. Taking advantage of the rapid process in the field of machine learning, deep neural networks (DNNs) have been extensively applied for the personalisation of HRTFs [1,2]. Such DNNs can, for example, be trained to directly predict the log-magnitude spectrum of HRTFs from edge-detected single-view ear images and individual anthropometric features [3].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Comparison of deep-neural-network architectures for the prediction of head-related transfer functions using a parametric pinna model

Pausch,

Perfler,

Holighaus

et al. 2024

Proceedings of the 10th Convention of the European Acoustics Association Forum Acusticum 2023

View full text Add to dashboard Cite

Personalised head-related transfer functions (HRTFs) represent a key component for applications in virtual or augmented reality with high demands in perceptual audio. Numerical methods enable the calculation of personalised HRTFs based on the individual geometry of a listener. Such a geometry can be generated by exploiting the information from multi-view-plus-depth (MVPD) images of the listener's pinna. However, the results are typically noisy, especially in regions relevant for HRTF calculations. To address this shortcoming, HRTFs can also be calculated based on a mesh obtained from a parametric pinna model (PPM) whose parameters were optimised to fit noisy measurements of an individual ear geometry. The feasibility of this approach has been investigated by employing a convolutional neural network (CNN) which predicts the PPMs parameters from synthetic MVPDs pinnaonly images. In this contribution, we varied the type of network architecture and analysed the effects on prediction accuracy. The results of comparative evaluations will be discussed in the geometric domain to explore the limits of the PPM-based HRTF personalisation, and corroborate the choice of a feasible network architecture before implementing further optimisation.

show abstract

Section: Network Architecturesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Comparison of deep-neural-network architectures for the prediction of head-related transfer functions using a parametric pinna model

Pausch,

Perfler,

Holighaus

et al. 2024

Proceedings of the 10th Convention of the European Acoustics Association Forum Acusticum 2023

View full text Add to dashboard Cite

show abstract

“…Recently, data-driven methods for binaural synthesis have been widely studied [16,17,18,19]. These methods inherently require a sufficiently large number of data points to estimate the distribution of interest.…”

Section: Introductionmentioning

confidence: 99%

Global HRTF Interpolation via Learned Affine Transformation of Hyper-conditioned Features

Lee¹,

Lee²,

Lee³

2022

Preprint

View full text Add to dashboard Cite

Estimating Head-Related Transfer Functions (HRTFs) of arbitrary source points is essential in immersive binaural audio rendering. Computing each individual's HRTFs is challenging, as traditional approaches require expensive time and computational resources, while modern data-driven approaches are data-hungry. Especially for the data-driven approaches, existing HRTF datasets differ in spatial sampling distributions of source positions, posing a major problem when generalizing the method across multiple datasets. To alleviate this, we propose a deep learning method based on a novel conditioning architecture. The proposed method can predict an HRTF of any position by interpolating the HRTFs of known distributions. Experimental results show that the proposed architecture improves the model's generalizability across datasets with various coordinate systems. Additional demonstrations using coarsened HRTFs demonstrate that the model robustly reconstructs the target HRTFs from the coarsened data.

show abstract

A Metaverse: Taxonomy, Components, Applications, and Open Challenges

2022

View full text Add to dashboard Cite

Unlike previous studies on the Metaverse based on Second Life, the current Metaverse is based on the social value of Generation Z that online and offline selves are not different. With the technological development of deep learning-based high-precision recognition models and natural generation models, Metaverse is being strengthened with various factors, from mobile-based always-on access to connectivity with reality using virtual currency. The integration of enhanced social activities and neural-net methods requires a new definition of Metaverse suitable for the present, different from the previous Metaverse. This paper divides the concepts and essential techniques necessary for realizing the Metaverse into three components (i.e., hardware, software, and contents) and three approaches (i.e., user interaction, implementation, and application) rather than marketing or hardware approach to conduct a comprehensive analysis. Furthermore, we describe essential methods based on three components and techniques to Metaverse's representative Ready Player One, Roblox, and Facebook research in the domain of films, games, and studies. Finally, we summarize the limitations and directions for implementing the immersive Metaverse as social influences, constraints, and open challenges.

show abstract

On the Predictability of Hrtfs from Ear Shapes Using Deep Networks

Cited by 10 publications

References 17 publications

Comparison of deep-neural-network architectures for the prediction of head-related transfer functions using a parametric pinna model

Comparison of deep-neural-network architectures for the prediction of head-related transfer functions using a parametric pinna model

Global HRTF Interpolation via Learned Affine Transformation of Hyper-conditioned Features

A Metaverse: Taxonomy, Components, Applications, and Open Challenges

Contact Info

Product

Resources

About