HRTF Performance Evaluation: Methodology and Metrics for Localisation Accuracy and Learning Assessment

Abstract

Through a review of the current literature, this chapter defines a methodology for the analysis of HRTF localisation performance, as applied to assess the quality of an HRTF selection or learning program. A case study is subsequently proposed, applying this methodology to a cross-comparison on the results of five contemporary experiments on HRTF learning. The objective is to propose a set of steps and metrics to allow for a systematic assessment of participant performance (baseline, learning rates, foreseeable performance plateau limits, etc.) to ease future inter-study comparisons.

When HRTF training doesn't work

“…We examine this evolution from L1 to reduce the contribution of procedural learning which appears most prevalent between L0 and L1. Previous studies results suggest that a certain percentage of non-proficient learners should be expected in every HRTF learning experiment [6,7,13]. This percentage is well above zero however, even in non-favorable conditions as when par- ticipants train with a worst-match HRTFs (e.g.…”

“…If there is a silver lining to the absence of perceptual HRTF adaptation reported after L1, it is that the evolu-tion of localization metrics observed between L0 and L1 might then be used as a measure of expected procedural learning improvement in those metrics. Based on the existing literature [13], the L0-L1 evolution leading to the L1-L3 performance plateau seems indeed both too fast and not large enough to be attributed to perceptual learning. If so, typical procedural learning evolution of those metrics are notably 5 • improvement in overall great-circle angle error, 7 % improvement in precision response rate, 4 % in in-cone error rate, and 2 % in off-cone error rates.…”

“…Localization accuracy was assessed based on the methodology presented in [13]. The global extent of the localization error is first assessed using the great-circle error.…”

Impact of wearing a head-mounted display on localization accuracy of real sound sources

“…front-back or up-down confusion rates, were calculated and compared across the aforementioned factors based on the polar angle classification scheme described in Zagala et al [17]. As for the polar error, the scheme was altered to avoid compression at the poles inflating up-down and front-back rates [14,18]: excluding targets with absolute lateral angle above 67.5°(within a 45°cone from the poles) from the analysis. Generalized linear mixed models (GLMMs), constructed as repeated measures logistic regressions [15], were used to evaluate the effects of HMD, localization task experience, and the interaction term on the percentage of reversal errors.…”

System-to-User and User-to-System Adaptations in Binaural Audio

“…It is clear that, while various methods and tools are available for selecting a best fit HRTF for a given listener, there is no established evaluation protocol to determine how well these methods work and compare with each other. While some work is advancing in proposing common methodologies and metrics [75], the lack of established methods raises some very relevant questions about the feasibility of a unique HRTF selection task which performs reliably and independently from factors such as the listeners expertise, the signals employed, the user interface, the context where the tests are carried out and, more in general, the task for which the final quality is judged. It seems evident that any major leap forward in this field is limited until two primary issues are addressed: (1) the establishment of pertinent metrics to perceptually assess HRTFs and (2) the relationship between these metrics and specific characteristics of the signal domain HRTF filters.…”