Another Look at Observational Studies in Rehabilitation Research: Going Beyond the Holy Grail of the Randomized Controlled Trial

Horn, Susan D.; DeJong, Gerben; Ryser, David K.; Teraoka, Jeffrey K.

doi:10.1016/j.apmr.2005.08.116

Cited by 129 publications

(103 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The specific aim of our study, however, was to investigate inter-therapist reliability among MDT trained clinicians for identifying diverse treatment techniques following PBE research criteria. 26,31,33,34,44 Researchers need to perform addition studies to assess how generalizable these reliability estimates are for larger samples of therapists with diverse post-graduate training. Because we are collaborating with therapists in Israel in our PBE studies, we are initiating a larger inter-rater reliability study to assess recognition of treatments outlined in Appendix among 400 physical therapists in Israel, including 168 therapists at different levels of MDT training from which 20 therapists have recently achieved a credential MDT level.…”

Section: Limitationsmentioning

confidence: 99%

“…26 Inter-rater reliability data facilitates the ultimate goal of PBE to identify which intervention(s) used by physical therapists during everyday routine care, are associated with better outcomes. 25,26,[31][32][33][34] In order to progress with PBE design studies investigating the McKenzie system, the importance of recording interventions applied by MDT therapists in a reliable and valid manner needs to be demonstrated. Explaining variance in outcomes using interventions as predictors in CER models can not begin until we can confirm that clinicians can identify and document interventions in a reliable way.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Clinician’s ability to identify neck and low back interventions: an inter-rater chance-corrected agreement pilot study

Werneke

Hart²,

Deutscher

et al. 2011

Journal of Manual & Manipulative Therapy

View full text Add to dashboard Cite

Objective: To estimate inter-rater agreement of physical therapists trained in MDT approach and participating in practice-based evidence (PBE) research to identify 72 physical therapy interventions in video demonstrations on a single model and clinical vignettes. PBE is a well designed observational study and demonstrating clinician observational consistency is an important step in conducting PBE research design. Methods: Two physical therapists volunteered to participate in pilot reliability testing and seven other physical therapists trained in McKenzie Mechanical Diagnosis and Therapy (MDT) methods volunteered for the inter-rater chance-corrected agreement study. All therapists identified interventions presented within 52 videos and 5 written clinical vignettes describing 20 more intervention techniques. Therapists independently identified all interventions. We assessed inter-rater chance-corrected agreement of therapists' ability to identify intervention techniques using Kappa coefficients with associated 95% confidence intervals and indices for bias and prevalence. Results: Of the 147 kappa coefficients estimated, 7% were (0.6, 10% were .0.6 and (0.8, and 83% were .0.8. Agreement was lowest for identifying cognitive behavioral techniques (median kappa50.79). The minimum and maximum prevalence and bias indices were 0.33 and 0.85 and 0 and 0.33, respectively suggesting kappa coefficient estimates were strong. Generalized kappa coefficients ranged from 0.73 to 1.00. Discussion: Results provide evidence that substantial to almost perfect inter-rater agreement could be expected when trained therapists identify physical therapy interventions used for patients with spinal impairments from staged videos and vignettes. This may be helpful to reassure clinicians of the quality of the reporting of intervention(s) performed when conducting multivariable analyses in future pragmatic PBE studies. Additional studies are needed to test whether these results can be validated using larger groups of therapists, trained and not trained in MDT methods, as well as examining different methods to examine inter-rater agreement for identifying diverse interventions commonly used for managing patients during routine practice.

show abstract

Section: Limitationsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Clinician’s ability to identify neck and low back interventions: an inter-rater chance-corrected agreement pilot study

Werneke

Hart²,

Deutscher

et al. 2011

Journal of Manual & Manipulative Therapy

View full text Add to dashboard Cite

show abstract

“…Physical therapists are usually aware of the treatment they are delivering (rater bias); blinding the patient with sham interventions may be difficult or impossible. 29 Additionally, most current Institutional Review Boards require that patients are aware of all of the possible interventions they may receive as part of the informed consent process, which provides another barrier to complete patient blinding. However, the person measuring outcomes in physical therapy trials can almost always be blinded to group assignment in order to minimize rater bias.…”

Section: Hierarchy Of Evidencementioning

confidence: 99%

A Primer on Selected Aspects of Evidence-Based Practice Relating to Questions of Treatment, Part 1: Asking Questions, Finding Evidence, and Determining Validity

Cleland

Noteboom²,

Whitman³

et al. 2008

J Orthop Sports Phys Ther

View full text Add to dashboard Cite

“…Undoubtedly, RCTs are the gold standard for assessing the efficacy of a given treatment; however, in reality, these high-level investigations are difficult to realise and can sometimes create ethical concerns [7]. In such circumstances, observational studies can offer a suitable alternative or complement to RCTs, contributing to the overall body of evidence [3,8].…”

mentioning

confidence: 99%

The changing face of quality in spine surgery

Grob

Mannion

2009

Eur Spine J

View full text Add to dashboard Cite

Another Look at Observational Studies in Rehabilitation Research: Going Beyond the Holy Grail of the Randomized Controlled Trial

Cited by 129 publications

References 33 publications

Clinician’s ability to identify neck and low back interventions: an inter-rater chance-corrected agreement pilot study

Clinician’s ability to identify neck and low back interventions: an inter-rater chance-corrected agreement pilot study

A Primer on Selected Aspects of Evidence-Based Practice Relating to Questions of Treatment, Part 1: Asking Questions, Finding Evidence, and Determining Validity

The changing face of quality in spine surgery

Contact Info

Product

Resources

About