Could the Trustworthiness in RAndomized Controlled Trials (TRACT) checklist be trusted?

Abstract

Randomized controlled trials (RCTs) are considered the gold standard study design in clinical effectiveness research, albeit some RCTs have been found to be fraudulent post-publication. The Trustworthiness in RAndomized Controlled Trials (TRACT) checklist examines RCT integrity through an assessment covering seven domains. While the authors claim that it can identify problematic trials, it has never been validated. A critical appraisal of the checklist highlights major failures in terms of health measurement methodology, including the arbitrary nature of items chosen. We applied the checklist to 16 RCTs coauthored by the checklist developer, and identified potential violations related to retrospective registration, ethical implications, authorship, methodology, implausible timeframes, zero participants to follow-up, implausible baseline characteristics, and excessive inter-group within trial as well as inter-trial similarities. In addition, we identified inconsistencies between the trials’ registration and data in the final published manuscripts. If we uncritically assume that the checklist is valid, then a formal institutional investigation into the trial portfolio its developer is warranted; especially targeting these 16 trials objectively shown to be problematic. The checklist should be considered as neither investigative nor conclusive of potential problems with RCTs until it is further refined and validated.