Inter-rater reliability of a modified version of Delitto et al.’s classification-based system for low back pain: a pilot study

Abstract

Study design: Observational inter-rater reliability study. Objectives: To examine: (1) the inter-rater reliability of a modified version of Delitto et al.'s classificationbased algorithm for patients with low back pain; (2) the influence of different levels of familiarity with the system; and (3) the inter-rater reliability of algorithm decisions in patients who clearly fit into a subgroup (clear classifications) and those who do not (unclear classifications). Methods: Patients were examined twice on the same day by two of three participating physical therapists with different levels of familiarity with the system. Patients were classified into one of four classification groups. Raters were blind to the others' classification decision. In order to quantify the inter-rater reliability, percentages of agreement and Cohen's Kappa were calculated. Results: A total of 36 patients were included (clear classification n523; unclear classification n513). The overall rate of agreement was 53% and the Kappa value was 0.34 [95% confidence interval (CI): 0.11-0.57], which indicated only fair inter-rater reliability. Inter-rater reliability for patients with a clear classification (agreement 52%, Kappa value 0.29) was not higher than for patients with an unclear classification (agreement 54%, Kappa value 0.33). Familiarity with the system (i.e. trained with written instructions and previous research experience with the algorithm) did not improve the inter-rater reliability. Conclusion: Our pilot study challenges the inter-rater reliability of the classification procedure in clinical practice. Therefore, more knowledge is needed about factors that affect the inter-rater reliability, in order to improve the clinical applicability of the classification scheme.

Prevalence and reliability of treatment-based classification for subgrouping patients with low back pain

“…Measuring reliability and agreement between raters for the TBC assessment is crucial in order to compare measurements between therapists, observe classification patterns and consistency, and also explore disagreements among them, contributing to the classification methods' further enhancement. This study presented a substantial agreement (k = 0.62) between raters, conflicting with previous studies by Apledoorn et al (2016) [41], Fritz et al…”

The Use of Cluster Analysis by Partitioning around Medoids (PAM) to Examine the Heterogeneity of Patients with Low Back Pain within Subgroups of The Treatment Based Classification System

“…If we consider the PAM analysis as a valid classification method, this relatively low agreement revealed the weakness of the 2007 TBC system. A moderate [23][24][25][26] and moderate to good [21,29,38,39] inter-observer reliability have been reported for the TBC system. To the authors' knowledge, this is the first study that examine the TBC classification using clustering by PAM analysis.…”

“…Stanton et al showed that the TBC criteria do not guarantee that patients comply with a single intervention, but that 25% of patients can meet criteria for more than one subgroup [ 23 ]. In addition, the prevalence of patients with “unclear classification” in early versions of TBC system has been estimated between 26 and 50 percent [ 23 , 24 , 38 , 40 ]. The developers of the 2007 TBC system also stated that this system had its limitations and needed to refine the algorithm [ 22 ].…”

“…Since, all variables in the TBC system were examined individually and separately, the behavior of one variable in the presence or absence of the others, may affect the final outcome of the classification. The CA approach in our study may be contrasted against traditional multivariate methods [21,[23][24][25][26][38][39][40][41] that typically analyze for significant differences between variables. In many applied research studies, the most useful level of analysis involves the consideration of the entire case more than comparisons between variables within each case [1].…”

“…This system, like other classification approaches, does not cover all patients with LBP and 25% of patients remained unclassified [23]. Some previous studies have reported moderate interobserver reliability for this system [23][24][25][26].…”