Standardizing assessment of spoken discourse in aphasia: A working group with deliverables

Abstract

Purpose: The heterogeneous nature of measures, methods, and analyses reported in the aphasia spoken discourse literature precludes comparison of outcomes across studies (e.g., meta-analyses) and inhibits replication. Furthermore, funding and time constraints significantly hinder collecting test-retest data on spoken discourse outcomes. This research note describes the development and structure of a working group, designed to address major gaps in the spoken discourse aphasia literature, including a lack of standardization in methodology, analysis, and reporting, as well as nominal data regarding the psychometric properties of spoken discourse outcomes.Method: The initial initiatives for this working group are to: 1) propose recommendations regarding standardization of spoken discourse collection, analysis, and reporting in aphasia, based on the results of an international survey and a systematic literature review; and, 2) create a database of test-retest spoken discourse data from individuals with and without aphasia. The survey of spoken discourse collection, analysis, and interpretation procedures was distributed to clinicians and researchers involved in aphasia assessment and rehabilitation September – November 2019. We will publish survey results and recommend standards for collecting, analyzing, and reporting spoken discourse in aphasia. A multisite endeavor to collect test-retest spoken discourse data from individuals with and without aphasia will be initiated. This test-retest information will be contributed to a central site for transcription and analysis, and data will be subsequently openly curated.Conclusion: The goal of the working group is to create recommendations for field-wide standards in methods, analysis and reporting of spoken discourse outcomes, as has been done across other related disciplines (e.g., CONSORT, EQUATOR, COBIDAS). Additionally, the creation of a database through our multi-site collaboration will allow the identification of psychometrically sound outcome measures and norms that can be used by clinicians and researchers to assess spoken discourse abilities in aphasia.

Leveraging big data to understand the interaction of task and language during monologic spoken discourse in speakers with and without aphasia

“…Our work and others [Wright & Capilouto, 2009, Stark, 2019 provide evidence for the establishment of fidelity during all stages of language sampling, from acquisition to transcription to coding to analysis. A recently founded working group, FOQUSAphasia ('FOstering QUality of Spoken discourse research in Aphasia; www.foqusaphasia.com; [Stark et al, 2020]) has endeavored to create initiatives to foster fidelity and standardization for discourse in aphasia. Without having these standards, it is difficult to reproduce or replicate prior findings in the field, and to draw conclusions (e.g., meta-analyses, scoping reviews) about spoken discourse across a variety of settings (e.g., assessment, therapy).…”

Leveraging big data to understand the interaction of task and language during monologic spoken discourse in speakers with and without aphasia

“…Our work and others [Wright & Capilouto, 2009, Stark, 2019 provide evidence for the establishment of fidelity during all stages of language sampling, from acquisition to transcription to coding to analysis. A recently founded working group, FOQUSAphasia ('FOstering QUality of Spoken discourse research in Aphasia; www.foqusaphasia.com; [Stark et al, 2020]) has endeavored to create initiatives to foster fidelity and standardization for discourse in aphasia. Without having these standards, it is difficult to reproduce or replicate prior findings in the field, and to draw conclusions (e.g.…”

Spoken Discourse Assessment and Analysis in Aphasia: An International Survey of Current Practices

“…Like the Committee on Best Practice in Data Analysis and Sharing (COBIDAS), created by the Organization for Human Brain Mapping (Nichols et al, 2017), the objective of creating and maintaining reporting standards for spoken discourse in aphasia is not to operationally define 'reproducibility' or 'replicability' in discourse analyses, but instead to identify practices that maximize analytical stability and generalizability of study findings. Given the growth in spoken discourse research in aphasia across the past few decades (e.g., Bryant et al, 2016), we propose that the creation of reporting standards will: "(a) encourage replication of studies; (b) ensure consistent reporting across studies; (c) recommend appropriate statistical modeling, thereby ensuring the most appropriate statistical inferences; and, (d) overall, contribute to a more homogeneous, rigorous and standardized process by which spoken discourse research is evaluated and ultimately disseminated for clinical use" (p. 6;Stark et al, 2020). Reporting standards (and indeed, adherence to set standards) will enable meta-analytic consolidation of evidence and more importantly, will have downstream, direct clinical implications by improving practices for collecting, analyzing, and accurately interpreting initial presenting status as well as changes in spoken discourse outcomes in aphasia.…”

“…Reporting standards (and indeed, adherence to set standards) will enable meta-analytic consolidation of evidence and more importantly, will have downstream, direct clinical implications by improving practices for collecting, analyzing, and accurately interpreting initial presenting status as well as changes in spoken discourse outcomes in aphasia. Accordingly, to address the major gaps in the existing literature and improve the state of research in spoken discourse in aphasia, the FOQUSAphasia (FOstering the QUality of Spoken discourse in Aphasia) working group was created (Stark et al, 2020;www.foqusaphasia.com). This working group comprises researchers, clinicians, and other stakeholders, including PWA and their caregivers.…”

“…The current survey was completed as part of the Best Practices Task Force within the FOQUSAphasia working group (Stark et al, 2020). The broad goal of the study was to survey current researchers and clinicians as a first step toward creating recommendations for field-wide standards in methods, analysis, and reporting of spoken discourse outcomes, as has been done across other related disciplines (Nichols et al, 2017;Simmons-Mackie et al, 2017;Wallace et al, 2019).…”