The reporting characteristics of bovine respiratory disease clinical intervention trials published prior to and following publication of the REFLECT statement

Abstract

The goal of the REFLECT Statement (Reporting guidElines For randomized controLled trials in livEstoCk and food safeTy) (published in 2010) was to provide the veterinary research community with reporting guidelines tailored for randomized controlled trials for livestock and food safety. Our objective was to determine the prevalence of REFLECT Statement reporting of items 1-19 in controlled trials published in journals between 1970 and 2017 examining the comparative efficacy of FDA-registered antimicrobials against naturally acquired BRD (bovine respiratory disease) in weaned beef calves in Canada or the USA, and to compare the prevalence of reporting before and after 2010, when REFLECT was published. We divided REFLECT Statement, items 3, 5, 10, and 11 into subitems, because each dealt with multiple elements requiring separate assessment. As a result, 28 different items or subitems were evaluated independently. We searched MEDLINE and CABI (CAB Abstracts and Global Health) (Web of Science™) in April 2017 and screened 2327 references. Two reviewers independently assessed the reporting of each item and subitem. Ninety-five references were eligible for the study. The reporting of the REFLECT items showed a point estimate for the prevalence ratio >1 (i.e. a higher proportion of studies published post-2010 reported this item compared to studies published pre-2010), apart from items 10.3, i.e., item 10, subitem 3 (who assigned study units to the interventions), 13 (the flow of study units through the study), 16 (number of study units in analysis), 18 (multiplicity), and 19 (adverse effects). Fifty-three (79%) of 67 studies published before 2010 and all 28 (100%) papers published after 2010 reported using a random allocation method in either the title, abstract, or methods (Prevalence ratio = 1.25; 95% CI (1.09,1.43)). However, 8 studies published prior to 2010 and 7 studies published post-2010 reported the term "systematic randomization" or variations of this term (which is not true randomization) to describe the allocation procedure. Fifty-five percent (37/67) of studies published pre-2010 reported blinding status (blinded/not blinded) of outcome assessors, compared to 24/28 (86%) of studies published post-2010 (Prevalence ratio = 1.5, 95% CI (1.19, 2.02)). The reporting of recommended items in journal articles in this body of work is generally improving; however, there is also evidence of confusion about what constitutes a random allocation procedure, and this suggests an educational need. As this study is observational, this precludes concluding that the publication of the REFLECT Statement was the cause of this trend.

Invited review: Completeness of reporting of experiments: REFLECTing on a year of animal trials in the Journal of Dairy Science

“…Details on intervention administration is mentioned in the Journal's Instructions for Authors document, which may be why a greater degree of compliance with this item was seen. This was similar to a review of reporting of trials on bovine respiratory disease (Totton et al, 2018), in which 93 % of the studies published after 2010 reported this item. Lack of compliance with this item has also been shown to be associated with reporting positive treatment effects (Sargeant et al, 2009a); it is encouraging that this item appears to be well reported in our sample.…”

“…These change to limit the scope of the study were made to aid in clarity of communication of the results. To determine the number of papers to assess, we calculated sample size for estimation of a proportion (Dohoo et al 2010) was based on an estimated prevalence of 40 % of included papers reporting a sample size justification (based on results of Totton et al, 2018) with an acceptable error (precision) of 10 %, resulting in a minimum of 105 papers.…”

“…The REFLECT checklist is included as supplementary Table 1. Reporting assessment was based on the survey questions designed by Totton et al (2018), where items 1 and 3 to 19 (of 22) in REFLECT were rephrased into questions (Table 2).…”

“…In addition to the subdivision of items 3, 5, 10, and 11 by Totton et al (2018), we also subdivided items 6, 7, and 8. Items 3, 5, 10, and 11 were split by the previous authors as they concerned more than one piece of information, while items 6 and 7 were subdivided as it was expected that few studies would have reported sample size justification, and item 8 was modified to include a question on random allocation reported elsewhere in the manuscript, and to capture those reporting a method of study unit allocation which was not random (e.g.…”

“…While there is some evidence that reporting has improved since the publication of reporting guidelines for human and animal trials, there is still much room for advancement, as many key features remain under-reported (Plint et al, 2006;Totton et al, 2018). Researchers may be unaware that reporting guidelines exist.…”