Inter-rater and intra-rater reliability and agreement of echocardiographic diagnosis of rheumatic heart disease using the World Heart Federation evidence-based criteria

Abstract

ObjectiveDifferent definitions have been used for screening for rheumatic heart disease (RHD). This led to the development of the 2012 evidence-based World Heart Federation (WHF) echocardiographic criteria. The objective of this study is to determine the intra-rater and inter-rater reliability and agreement in differentiating no RHD from mild RHD using the WHF echocardiographic criteria.MethodsA standard set of 200 echocardiograms was collated from prior population-based surveys and uploaded for blinded web-based reporting. Fifteen international cardiologists reported on and categorised each echocardiogram as no RHD, borderline or definite RHD. Intra-rater and inter-rater reliability was calculated using Cohen’s and Fleiss’ free-marginal multirater kappa (κ) statistics, respectively. Agreement assessment was expressed as percentages. Subanalyses assessed reproducibility and agreement parameters in detecting individual components of WHF criteria.ResultsSample size from a statistical standpoint was 3000, based on repeated reporting of the 200 studies. The inter-rater and intra-rater reliability of diagnosing definite RHD was substantial with a kappa of 0.65 and 0.69, respectively. The diagnosis of pathological mitral and aortic regurgitation was reliable and almost perfect, kappa of 0.79 and 0.86, respectively. Agreement for morphological changes of RHD was variable ranging from 0.54 to 0.93 κ.ConclusionsThe WHF echocardiographic criteria enable reproducible categorisation of echocardiograms as definite RHD versus no or borderline RHD and hence it would be a suitable tool for screening and monitoring disease progression. The study highlights the strengths and limitations of the WHF echo criteria and provides a platform for future revisions.

The variable spectrum of anterior mitral valve leaflet restriction in rheumatic heart disease screening

“…These reasons may go a long way to explain the findings of a recent study that evaluated the reproducibility of the WHF criteria. Here, WHF MV restriction, together with chordal thickening, was identified as one of the least reproducible WHF morphological criteria (κ = 0.55; 95% CI 0.49–0.60) 9 …”

“…Here, WHF MV restriction, together with chordal thickening, was identified as one of the least reproducible WHF morphological criteria (κ = 0.55; 95% CI 0.49-0.60). 9 In the present study, we sought to address these potential limitations within the WHF guideline by introducing a strict screening definition of AMVL restriction. Using this definition, we found an equal proportion and relatively high number of AMVL restriction cases in both the HP (67 cases, 11.6%) and VLP cohorts (35 cases, 9.7%).…”

Comparison of approaches to determine echocardiographic outcomes for children with latent rheumatic heart disease

“…Previous inter-reviewer reliability assessments have shown good-to-perfect agreement on presence of and diagnostic category of MR, but much less agreement on MV morphology features. Additionally, morphological criteria have repeatedly been found to be neither sensitive nor specific, with poor inter-reviewer reliability compared with functional assessment 6 16–19…”

A new perspective: Fat tissue and adipokines in rheumatic heart valves

“…After infection with hemolytic streptococcus, edema occurs at the base and edge of the heart valve, leading to inflammatory reaction and fibrosis, which leads to fusion of chordae tendontosa and leaflets at the valve junction, calcification, deformation, stiffness, and fibrosis of leaflets, and ultimately lead to valve insufficiency or stenosis 7 . Echocardiography is recommended by the World Heart Federation for the screening and diagnosis of RHD 8 . Through three‐dimensional volume measurement, the heart shape and structure can be truly stereoscopic, with good repeatability and high accuracy 9 .…”