Reproducibility in Cognitive Hearing Research: Theoretical Considerations and Their Practical Application in Multi-Lab Studies

Heinrich, Antje; Knight, Sarah

doi:10.3389/fpsyg.2020.01590

Cited by 7 publications

(6 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Setting aside this limitation, we could therefore envisage OMEXP to help with the homogenization of experiment design and data collection format in the cognitive hearing science, aiding researchers to better plan experiments and synthesize knowledge across experiments and sites (for instance, enabling meta-analysis and multi-centered studies) (Heinrich & Knight, 2020). We propose to start this homogenization effort with an OMEXP implementation of the widely used Hearing In Noise Test (HINT) (Nilsson, Soli, & Sullivan, 1994) adding concurrent pupillometry.…”

Section: Discussionmentioning

confidence: 99%

Streamlining experiment design in cognitive hearing science using OpenSesame

Sulas¹,

Hasan

Zhang³

et al. 2022

Behav Res

View full text Add to dashboard Cite

Auditory science increasingly builds on concepts and testing paradigms originated in behavioral psychology and cognitive neuroscience – an evolution of which the resulting discipline is now known as cognitive hearing science. Experimental cognitive hearing science paradigms call for hybrid cognitive and psychobehavioral tests such as those relating the attentional system, working memory, and executive functioning to low-level auditory acuity or speech intelligibility. Building complex multi-stimuli experiments can rapidly become time-consuming and error-prone. Platform-based experiment design can help streamline the implementation of cognitive hearing science experimental paradigms, promote the standardization of experiment design practices, and ensure reliability and control. Here, we introduce a set of features for the open-source python-based OpenSesame platform that allows the rapid implementation of custom behavioral and cognitive hearing science tests, including complex multichannel audio stimuli while interfacing with various synchronous inputs/outputs. Our integration includes advanced audio playback capabilities with multiple loudspeakers, an adaptive procedure, compatibility with standard I/Os and their synchronization through implementation of the Lab Streaming Layer protocol. We exemplify the capabilities of this extended OpenSesame platform with an implementation of the three-alternative forced choice amplitude modulation detection test and discuss reliability and performance. The new features are available free of charge from GitHub: https://github.com/elus-om/BRM_OMEXP.

show abstract

Section: Discussionmentioning

confidence: 99%

Streamlining experiment design in cognitive hearing science using OpenSesame

Sulas¹,

Hasan

Zhang³

et al. 2022

Behav Res

View full text Add to dashboard Cite

show abstract

“…Finally, the MD was calculated to define the smallest difference/change in outcome measure performance that can be considered to represent a “real” post-training difference. For a more detailed discussion of the choice of ICC, SEM, and MD in relation to hearing-related measures, see also Heinrich et al (2019 ) and Heinrich & Knight (2020 ).…”

Section: Methodsmentioning

confidence: 99%

Cogmed Training Does Not Generalize to Real-World Benefits for Adult Hearing Aid Users: Results of a Blinded, Active-Controlled Randomized Trial

et al. 2021

Self Cite

View full text Add to dashboard Cite

Objectives: Performance on working memory tasks is positively associated with speech-in-noise perception performance, particularly where auditory inputs are degraded. It is suggested that interventions designed to improve working memory capacity may improve domain-general working memory performance for people with hearing loss, to benefit their real-world listening. We examined whether a 5-week training program that primarily targets the storage component of working memory (Cogmed RM, adaptive) could improve cognition, speech-in-noise perception and self-reported hearing in a randomized controlled trial of adult hearing aid users with mild to moderate hearing loss, compared with an active control (Cogmed RM, nonadaptive) group of adults from the same population.Design: A preregistered randomized controlled trial of 57 adult hearing aid users (n = 27 experimental, n = 30 active control), recruited from a dedicated database of research volunteers, examined on-task learning and generalized improvements in measures of trained and untrained cognition, untrained speech-in-noise perception and self-reported hearing abilities, pre-to post-training. Participants and the outcome assessor were both blinded to intervention allocation. Retention of training-related improvements was examined at a 6-month follow-up assessment.Results: Per-protocol analyses showed improvements in trained tasks (Cogmed Index Improvement) that transferred to improvements in a trained working memory task tested outside of the training software (Backward Digit Span) and a small improvement in self-reported hearing ability (Glasgow Hearing Aid Benefit Profile, Initial Disability subscale). Both of these improvements were maintained 6-month post-training. There was no transfer of learning shown to untrained measures of cognition (working memory or attention), speech-in-noise perception, or self-reported hearing in everyday life. An assessment of individual differences showed that participants with better baseline working memory performance achieved greater learning on the trained tasks. Posttraining performance for untrained outcomes was largely predicted by individuals' pretraining performance on those measures.Conclusions: Despite significant on-task learning, generalized improvements of working memory training in this trial were limited to (a) improvements for a trained working memory task tested outside of the training software and (b) a small improvement in self-reported hearing ability for those in the experimental group, compared with active controls. We found no evidence to suggest that training which primarily targets storage aspects of working memory can result in domain-general improvements that benefit everyday communication for adult hearing aid users. These findings are consistent with a significant body of evidence showing that Cogmed training only improves performance for tasks that resemble Cogmed training. Future research should focus on the benefits of interventions that enhance cognition in the context in which it is employed withi...

show abstract

“…As discussed in the introduction, performance on many cognitive tasks weakly correlates with speech recognition accuracy (Dryden et al 2017), so it is generally unsurprising to find positive correlations such as the ones shown in Table 1 and Figure 2 so long as the tasks used have sufficient reliability (Parsons et al 2019;Heinrich & Knight 2020). Latent factor modeling demonstrated that all three tasks used here measure the latent memory factor which was related to sentence recognition to some extent, so it appears that observing associations between memory and sentence recognition does not depend on the use of a specific memory task.…”

Section: When Does Memory Affect Speech Recognition In Young Adults W...mentioning

confidence: 99%

Identifying Links Between Latent Memory and Speech Recognition Factors

Bosen,

Doria

2023

Ear &Amp; Hearing

View full text Add to dashboard Cite

Objectives: The link between memory ability and speech recognition accuracy is often examined by correlating summary measures of performance across various tasks, but interpretation of such correlations critically depends on assumptions about how these measures map onto underlying factors of interest. The present work presents an alternative approach, wherein latent factor models are fit to trial-level data from multiple tasks to directly test hypotheses about the underlying structure of memory and the extent to which latent memory factors are associated with individual differences in speech recognition accuracy. Latent factor models with different numbers of factors were fit to the data and compared to one another to select the structures which best explained vocoded sentence recognition in a two-talker masker across a range of target-to-masker ratios, performance on three memory tasks, and the link between sentence recognition and memory. Design: Young adults with normal hearing (N = 52 for the memory tasks, of which 21 participants also completed the sentence recognition task) completed three memory tasks and one sentence recognition task: reading span, auditory digit span, visual free recall of words, and recognition of 16-channel vocoded Perceptually Robust English Sentence Test Open-set sentences in the presence of a two-talker masker at target-to-masker ratios between +10 and 0 dB. Correlations between summary measures of memory task performance and sentence recognition accuracy were calculated for comparison to prior work, and latent factor models were fit to trial-level data and compared against one another to identify the number of latent factors which best explains the data. Models with one or two latent factors were fit to the sentence recognition data and models with one, two, or three latent factors were fit to the memory task data. Based on findings with these models, full models that linked one speech factor to one, two, or three memory factors were fit to the full data set. Models were compared via Expected Log pointwise Predictive Density and post hoc inspection of model parameters. Results: Summary measures were positively correlated across memory tasks and sentence recognition. Latent factor models revealed that sentence recognition accuracy was best explained by a single factor that varied across participants. Memory task performance was best explained by two latent factors, of which one was generally associated with performance on all three tasks and the other was specific to digit span recall accuracy at lists of six digits or more. When these models were combined, the general memory factor was closely related to the sentence recognition factor, whereas the factor specific to digit span had no apparent association with sentence recognition. Conclusions: Comparison of latent factor models enables testing hypotheses about the underlying structure linking cognition and speech recognition. This approach showed that multiple memory tasks assess a common latent factor that is related to individual differences in sentence recognition, although performance on some tasks was associated with multiple factors. Thus, while these tasks provide some convergent assessment of common latent factors, caution is needed when interpreting what they tell us about speech recognition.

show abstract

Reproducibility in Cognitive Hearing Research: Theoretical Considerations and Their Practical Application in Multi-Lab Studies

Cited by 7 publications

References 67 publications

Streamlining experiment design in cognitive hearing science using OpenSesame

Streamlining experiment design in cognitive hearing science using OpenSesame

Cogmed Training Does Not Generalize to Real-World Benefits for Adult Hearing Aid Users: Results of a Blinded, Active-Controlled Randomized Trial

Identifying Links Between Latent Memory and Speech Recognition Factors

Contact Info

Product

Resources

About