Bayesian Analysis of Phoneme Confusion Matrices

Abstract

This paper presents a parametric Bayesian approach to the statistical analysis of phoneme confusion matrices measured for groups of individual listeners in one or more test conditions. Two different bias problems in conventional estimation of mutual information are analyzed and explained theoretically. Evaluations with synthetic datasets indicate that the proposed Bayesian method can give satisfactory estimates of mutual information and response probabilities, even for phoneme confusion tests using a very small number of test items for each phoneme category. The proposed method can reveal overall differences in performance between two test conditions with better power than conventional Wilcoxon significance tests or conventional confidence intervals. The method can also identify sets of confusion-matrix cells that are credibly different between two test conditions, with better power than a similar approximate frequentist method.

Comparison of Frequency Transposition and Frequency Compression for People With Extensive Dead Regions in the Cochlea

“…This is consistent with previous research using frequency lowering (Alexander, 2016; Ellis & Munro, 2015; Glista et al., 2009; Kokx-Ryan et al., 2015; Posen et al., 1993; Robinson et al., 2007, 2009; Salorio-Corbetto et al., 2017a; Simpson et al., 2006). Bayesian statistical methods and tools developed by Leijon et al. (2016) were used to determine whether either FT or FC led to a change in the correct identification of any specific consonants.…”

“…A method for identifying differences with greater reliability than just selecting a threshold change in PC is needed. One such method is based on Bayesian analysis (Leijon, Henter, & Dahlquist, 2016). With this method, a prior probability of the response for each cell of the confusion matrix is assumed, based on a Dirichlet distribution (Leijon et al., 2016).…”

“…One such method is based on Bayesian analysis (Leijon, Henter, & Dahlquist, 2016). With this method, a prior probability of the response for each cell of the confusion matrix is assumed, based on a Dirichlet distribution (Leijon et al., 2016). This prior probability is updated to a posterior probability given the data.…”

“…If this probability ( p ) is small enough, the null hypothesis is rejected, and this is taken as indicating a significant difference across conditions. The Bayesian analysis, instead, gives the conditional probability ( q ) of the PC value for Condition A being better than that for Condition B, given the data, and the probability of the opposite event (i.e., that the PC value for Condition B is better than that for A, given the data), which is 1 − q (Leijon et al., 2016). In this article, we take q values ≥ 0.8 or ≤ 0.2 as indicating reliable differences across conditions.…”

“…Again, we adopted a threshold of Bayesian credibility ≥ 0.8 as indicating reliable differences across conditions. The Bayesian analysis of the consonant-confusion matrices was carried out using methods and software tools developed by Leijon et al. (2016) to determine whether FC or FT improved or worsened performance compared with Control and each other, for each participant and for the group.…”