Problems With Reliability of Speech Variables for Use in Quality Registries for Cleft Lip and Palate—Experiences From the Swedish Cleft Lip and Palate Registry

Malmborn, Jan Olof; Becker, Magnus; Klintö, Kristina

doi:10.1177/1055665618765777

Cited by 7 publications

(12 citation statements)

References 40 publications

(76 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the study by Brunnegård et al [29], agreement between judges for the percentage of correct consonants was excellent, with a singlemeasures intra-class coefficient of 0.85. Further, reliability of data on the percentage of non-oral errors in the CL/P registry was excellent in the study by Brunnegård et al, and in a previous study [31].…”

Section: Speech Datamentioning

confidence: 69%

Coverage, reporting degree and design of the Swedish quality registry for patients born with cleft lip and/or palate

Klintö

Karsten

Marcusson

et al. 2020

BMC Health Serv Res

Self Cite

View full text Add to dashboard Cite

Background: The objective of the Swedish cleft lip and palate (CLP) registry is to promote quality control, research and improvement of treatment, by comparison of the long-term results of surgery, orthodontics and speech from all six Swedish CLP centres. The purpose of the study was to investigate the coverage and reporting degree of the Swedish CLP registry, and to describe the design of the registry and discuss questions of reliability and validity of the data included. Methods: All six Swedish CLP centres participate in the registry. All children in Sweden with cleft lip and/or cleft palate, born from 2009 onwards, are included in the registry. Baseline data such as cleft type (ICD-10 diagnosis), heredity, birth weight and additional deformities and/or syndromes, as well as pre-surgical treatment, are recorded at first visit. Data on surgical treatment are recorded continuously. Treatment outcome regarding dentofacial development and speech are recorded at follow-ups at 5, 10, 16 and 19 years of age. Data on dentofacial development are also recorded 1 year after orthognathic surgery. In addition, data on babbling and speech are recorded at 18 months of age. Coverage degree and reporting degree of surgery was assessed by comparison with registrations in the Swedish Central patient registry. Reporting degree of orthodontic and speech registrations at 5 years of age was assessed by comparison with registrations at baseline. Results: The average coverage degree for children born 2009 to 2018 was 95.1%. For cleft-related surgeries, the average reporting degree was 92.4%. Average reporting degree of orthodontic registrations and speech registrations at age 5 years was 92 and 97.5% respectively. Conclusion: In order to achieve valid and reliable data in a healthcare quality registry, the degree of coverage and reporting needs to be high, the variables included should be limited and checked for reliability, and the professionals must calibrate themselves regularly. The Swedish CLP registry fulfils these requirements.

show abstract

Section: Speech Datamentioning

confidence: 69%

Coverage, reporting degree and design of the Swedish quality registry for patients born with cleft lip and/or palate

Klintö

Karsten

Marcusson

et al. 2020

BMC Health Serv Res

Self Cite

View full text Add to dashboard Cite

show abstract

“…Swedish is a language without nonoral consonants, such as glottal stops and active nasal fricatives. Thus, one possible explanation for the good agreement for nonoral errors observed by Malmborn et al (2018) is that few children displayed nonoral errors. The authors noted that if more children with nonoral errors had been included, the agreement might have been lower.…”

Section: Introductionmentioning

confidence: 87%

“…The guidelines for rating VPC were refined, as suggested by Malmborn et al (2018), and these changes were discussed and agreed upon by the group of SLPs working at the Swedish CLP centers. Marginally incompetent/insufficient VPC was defined as "mild symptoms in the speech, suggesting small problems with the velopharyngeal closure, which may be sufficient in some contexts/speech materials and somewhat insufficient in others, which did not lead to a recommendation about surgical treatment."…”

Section: Perceptual Reassessment By 4 Blinded Listenersmentioning

confidence: 99%

“…In the study by Malmborn et al (2018), agreement between registry data on VPC and results from reassessments was moderate, and the authors concluded that registry data on VPC should be interpreted with caution. Velopharyngeal competence is a validated measure with moderate-to-good agreement among trained listeners (Lohmander et al, 2017a; Lohmander et al, 2017b).…”

Section: Introductionmentioning

confidence: 97%

“…A recent pilot study compared speech data from the Swedish CLP registry with the results of reassessment by 3 blinded judges based on audio recordings of the same children (Malmborn et al, 2018). One shortcoming of this previous study is that speech data originated from 1 center only, which may have affected the results.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Reliability of Speech Variables and Speech-Related Quality Indicators in the Swedish Cleft Lip and Palate Registry

Brunnegård

Hagberg

Havstam

et al. 2020

The Cleft Palate Craniofacial Journal

Self Cite

View full text Add to dashboard Cite

Objective: To assess the reliability of speech variables and speech-related quality indicators in the Swedish quality registry for cleft lip and palate (CLP). Design: Retrospective study. Setting: Primary care university hospitals. Participants: Fifty-two 5-year-old children with unilateral CLP and 41 with bilateral CLP. Main Outcome Measures: Registry data for “percent nonoral errors” and “perceived velopharyngeal competence” (VPC) were compared to reassessments by 4 independent judges based on audio recordings. Interjudge agreement for “percent consonants correct” (PCC) and the reliability of 3 quality indicators were also assessed. Agreement was calculated with single measures intraclass correlation coefficient (ICC) for articulation outcomes, quadratic weighted κ and ICC for VPC, and percentage agreement and κ for quality indicators. Results: When the agreement between registry data and the judges’ reassessments was assessed, the ICC was 0.79 for percent nonoral errors. For VPC, the κ coefficient was 0.66 to 0.75 and the ICC was 0.73. Interjudge agreement for PCC calculated with ICC was 0.85. For the quality indicator “proportion of children with ≥86% correct consonants,” all 4 judges were in agreement for 72% of the cases. For “proportion of children without nonoral speech errors” and “proportion of children with competent or marginally incompetent velopharyngeal function,” the agreement between registry data and the 4 judges was 89% and 85%, respectively. Conclusions: The results indicate that registry data on PCC, percent nonoral errors, VPC, and the quality indicators “proportion of children without nonoral speech errors” and “proportion of children with competent or marginally incompetent velopharyngeal function” are reliable.

show abstract

Reliability of Outcome Measures to Assess Consonant Proficiency Following Cleft Palate Speech Intervention: The Percentage of Consonants Correct Metric and the Probe Scoring System

Alighieri

Bettens

Bruneel

et al. 2021

J Speech Lang Hear Res

View full text Add to dashboard Cite

Purpose This study compared the inter- and intrarater reliability of the percentage of consonants correct (PCC) metrics and the probe scoring system between an experienced and a less experienced rater and between two experienced raters. In addition, these outcome measures' ability to reflect changes following speech intervention was measured. Method During Phase 1, two raters (Rater 1 with 5 years of experience in cleft-related speech disorders and Rater 2 with limited experience in cleft-related speech disorders) independently assessed 134 speech samples at the word and sentence levels, which were collected on different data points before, during, and following a cleft palate speech intervention. During Phase 2, a third rater (with 8 years of experience) analyzed 34 speech samples. The percentage of consonants correct–revised, the percentage of correct places and manners, and probe scores at the word and sentence levels were measured. Results Poor-to-moderate interreliability between Raters 1 and 2 was found due to differences in error classification. Interrater reliability between Raters 1 and 3 was very good for both the PCC metrics and the probe scores. The interrater reliability for the amount of targets elicited was lower compared to the interrater reliability for the amount of targets correct. The probe scoring system demonstrated a greater ability to detect changes toward the correct production of the target consonant compared to the PCC metrics. Conclusions Having an experience with the assessment of cleft-related speech disorders is a crucial factor to gain reliable results. The interrater reliability for the PCC metrics and the probe scoring system between two experienced raters did not differ, suggesting that both outcome measures can be used in cleft palate speech intervention studies. Despite the ability of the probe scoring system to detect changes, further research should provide insight in the benefits of this system both for research and clinical purposes.

show abstract

Problems With Reliability of Speech Variables for Use in Quality Registries for Cleft Lip and Palate—Experiences From the Swedish Cleft Lip and Palate Registry

Cited by 7 publications

References 40 publications

Coverage, reporting degree and design of the Swedish quality registry for patients born with cleft lip and/or palate

Coverage, reporting degree and design of the Swedish quality registry for patients born with cleft lip and/or palate

Reliability of Speech Variables and Speech-Related Quality Indicators in the Swedish Cleft Lip and Palate Registry

Reliability of Outcome Measures to Assess Consonant Proficiency Following Cleft Palate Speech Intervention: The Percentage of Consonants Correct Metric and the Probe Scoring System

Contact Info

Product

Resources

About