Interobserver Agreement in the Application of Levels of Evidence to Scientific Papers in the American Volume of The Journal of Bone and Joint Surgery

Bhandari, Mohit; Swiontkowski, Marc F.; Einhorn, Thomas A.; Tornetta, Paul; Schemitsch, Emil H.; Leece, Pamela; Sprague, Sheila; Wright, James G.

doi:10.2106/00004623-200408000-00016

Cited by 100 publications

(65 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…An accurate identification rate (77% and 83%), with a low incidence of misclassification (4% to 5% leading to an inappropriate increase in quality) indicates that LOE is a concept with which faculty and residents are conversant. Our results are consistent with those of other studies that show substantial or better agreement when grading LOE, regardless of experience [2,4,17,32]. In contrast, Schmidt et al [21] found substantial inconsistency and poor agreement between different observers when grading the LOE of presentations at the 2007 AAOS Annual Meeting.…”

Section: Discussionsupporting

confidence: 91%

See 1 more Smart Citation

Have Levels of Evidence Improved the Quality of Orthopaedic Research?

Cunningham

Harmsen

Kweon

et al. 2013

Clinical Orthopaedics &Amp; Related Research

View full text Add to dashboard Cite

Background Since 2003 many orthopaedic journals have adopted grading systems for levels of evidence (LOE). It is unclear if the quality of orthopaedic literature has changed since LOE was introduced. Questions/purposes We asked three questions: (1) Have the overall number and proportion of Level I and II studies increased in the orthopaedic literature since the introduction of LOE? (2) Is a similar pattern seen in individual orthopaedic subspecialty journals? (3) What is the interobserver reliability of grading LOE? Methods We assigned LOE to therapeutic studies published in 2000, 2005, and 2010 in eight major orthopaedic subspecialty journals. Number and proportion of Level I and II publications were determined. Data were evaluated using log-linear models. Twenty-six reviewers (13 residents and 13 attendings) graded LOE of 20 blinded therapeutic articles from the Journal of Bone and Joint Surgery for 2009. Interobserver agreement relative to the Journal of Bone and Joint Surgery was assessed using a weighted kappa.

show abstract

Section: Discussionsupporting

confidence: 91%

“…Errors made by these observers could be reflected in the counts of articles analyzed for the study data. Based on the high interobserver reliability in our study and others [2,4,17,32], it is unlikely that this had a significant effect on our conclusions. Third, not all of the journals behaved similarly.…”

Section: Discussionsupporting

confidence: 38%

Have Levels of Evidence Improved the Quality of Orthopaedic Research?

Cunningham

Harmsen

Kweon

et al. 2013

Clinical Orthopaedics &Amp; Related Research

View full text Add to dashboard Cite

show abstract

“…All articles were reviewed by the two independent investigators for retrieval of the journal name, publication date, first and last authors, year of publication, geographic origin, total number of citations of the article, overall citation rate (total citations/article age), current citation rate (measured as the number of citations in the year 2013), research nature (sports traumatology, degenerative disease, scoring system, kinematic/biomechanical, imaging, or anatomic), and level of evidence (I through V for clinical articles, according to the guidelines specified in this journal 13 ). The two investigators disagreed regarding the characteristics of seventeen articles; these were reevaluated and discussed with a third investigator (S.K.…”

Section: Data Extractionmentioning

confidence: 99%

The Hundred Most-Cited Publications in Orthopaedic Knee Research

Ahmad

Evangelopoulos

Abbasian

et al. 2014

The Journal of Bone and Joint Surgery

131

View full text Add to dashboard Cite

show abstract

“…The LOE framework is a crude, yet reliable tool with which to rapidly categorize clinical studies based on their likelihood to be influenced by bias [4]. Since 2001, the LOE of clinical research presented at the AAOS and OTA annual meetings has increased [41,46].…”

Section: Discussionmentioning

confidence: 99%

Has the Level of Evidence of Podium Presentations at the Musculoskeletal Tumor Society Annual Meeting Changed Over Time?

Lerman

Cable

Thornley

et al. 2017

Clinical Orthopaedics &Amp; Related Research

Self Cite

View full text Add to dashboard Cite

Background Level of evidence (LOE) framework is a tool with which to categorize clinical studies based on their likelihood to be influenced by bias. Improvements in LOE have been demonstrated throughout orthopaedics, prompting our evaluation of orthopaedic oncology research LOE to determine if it has changed in kind. Questions/purposes (1) Has the LOE presented at the Musculoskeletal Tumor Society (MSTS) annual meeting improved over time? (2) Over the past decade, how do the MSTS and Orthopaedic Trauma Association (OTA) annual meetings compare regarding LOE overall and for the subset of therapeutic studies? Methods We reviewed abstracts from MSTS and OTA annual meeting podium presentations from 2005 to 2014. Three independent reviewers evaluated a total of 1222 abstracts for study type and LOE; there were 577 abstracts from MSTS and 645 from OTA. Changes in the distributions of study type and LOE over time were evaluated by Pearson chi-square test. Results There was no change over time in MSTS LOE for all study types (p = 0.13) and therapeutic (p = 0.36) study types during the reviewed decade. In contrast, OTA LOE increased over this time for all study types (p \ 0.01). The proportion of Level I therapeutic studies was higher at the OTA than the MSTS (3% [14 of 413] versus 0.5% [two of 387], respectively), whereas the proportion of Level IV studies was lower at the OTA than the MSTS (32% [134 of 413] versus 75% [292 of 387], respectively) during the reviewed decade. The proportion of controlled therapeutic studies (LOE I through III) versus uncontrolled studies (LOE IV) increased over time at OTA (p \ 0.021), but not at MSTS (p = 0.10). Conclusions Uncontrolled case series continue to dominate the MSTS scientific program, limiting progress in

show abstract

Interobserver Agreement in the Application of Levels of Evidence to Scientific Papers in the American Volume of The Journal of Bone and Joint Surgery

Cited by 100 publications

References 2 publications

Have Levels of Evidence Improved the Quality of Orthopaedic Research?

Have Levels of Evidence Improved the Quality of Orthopaedic Research?

The Hundred Most-Cited Publications in Orthopaedic Knee Research

Has the Level of Evidence of Podium Presentations at the Musculoskeletal Tumor Society Annual Meeting Changed Over Time?

Contact Info

Product

Resources

About