A Proposal for Structured Reporting of Randomized Controlled Trials

Introduction Military-connected youth are at increased risk for experiencing distinct psychosocial and behavioral health vulnerabilities. Although behavioral health interventions have been developed to treat vulnerabilities in military-connected youth, little is known about the methodological quality of studies evaluating these interventions. In this study, a systematic review of behavioral health interventions for military-connected youth was conducted to examine methodological quality and treatment outcomes. Materials and Methods Electronic databases were systematically searched for studies evaluating behavioral health interventions for military-connected youth which yielded 3,324 citations. Methodological quality was evaluated by 2 researchers with 3 measures that assessed scientific rigor, transparency, external and internal validity, and power for quantitative, qualitative, and mixed-method trials. Interrater reliability was strong (κ = 0.81). Sample characteristics and treatment outcomes were also assessed. Results Fourteen studies meeting full inclusion criteria evaluated 10 behavioral health interventions. Methodological quality scores for all studies were poor to fair, with limitations in reporting, external and internal validity, and power. Research designs were predominantly nonexperimental. Treatment effects for both psychosocial and behavioral health outcomes were consistently positive for all studies. In studies reporting effect sizes, treatment effects were small to moderate (d = 0.01–0.42, odds ratio = 0.04–0.47, b = −0.02–0.56). Demographic and military characteristics of samples were inconsistently reported. Conclusion Behavioral health interventions for military-connected youth have noteworthy methodological limitations, indicating a need to employ more rigorous research strategies. Positive treatment outcomes, however, suggest promising interventions for improving psychosocial and behavioral health problems in military-connected youth. Future research directions and implications for clinical-community practice are also discussed.

show abstract

“…10 The Quality Checklist has demonstrated good internal consistency reliability, 22 test-retest reliability, 27,28 and convergent validity. 29 …”

Section: Methodsmentioning

confidence: 99%

Evaluating Behavioral Health Interventions for Military-Connected Youth: A Systematic Review

et al. 2017

View full text Add to dashboard Cite

show abstract

“…However, it is important to remember that criterion (concurrent) validity is used to validate a tool in relation to a gold standard. In this case, the concurrent validity of the abovementioned scales was tested using the Chalmers, Delphi List, Jadad, CONSORT, Imperiale, Reisch, van Tulder, Standards of Reporting Trials Group (SRTG), 152 and European Lung Cancer Working Party 70 tools, which are not recognized as gold standards in this field. Based on this fact, the concurrent validity of the Jadad, Delphi List, Maastricht, Cho and Bero, Sindhu, Detsky, Downs and Black, Imperiale, Reisch, Chalmers, and van Tulder scales may be inappropriate.…”

Section: Assessing Quality Of Rctsmentioning

confidence: 99%

Scales to Assess the Quality of Randomized Controlled Trials: A Systematic Review

et al. 2008

View full text Add to dashboard Cite

Background and PurposeThe methodological quality of randomized controlled trials (RCTs) is commonly evaluated in order to assess the risk of biased estimates of treatment effects. The purpose of this systematic review was to identify scales used to evaluate the methodological quality of RCTs in health care research and summarize the content, construction, development, and psychometric properties of these scales. MethodsExtensive electronic database searches, along with a manual search, were performed. ResultsOne hundred five relevant studies were identified. They accounted for 21 scales and their modifications. The majority of scales had not been rigorously developed or tested for validity and reliability. The Jadad Scale presented the best validity and reliability evidence; however, its validity for physical therapy trials has not been supported. Discussion and ConclusionMany scales are used to evaluate the methodological quality of RCTs, but most of these scales have not been adequately developed and have not been adequately tested for validity and reliability. A valid and reliable scale for the assessment of the methodological quality of physical therapy trials needs to be developed. T he medical literature is an important resource to guide clinical decision making and research. The evaluation of the methodological quality of studies is an essential step in the process of selecting the best clinical literature. According to Verhagen et al, 1 assessment of methodological quality involves evaluation of internal validity (the degree to which a study's design, conduct, and analysis have minimized biases) and external validity (the extent to which the results of a study can be generalized outside the experimental situation) as well as statistical analysis of primary research. Taken together, these validity constructs are important in determining the methodological quality of primary research. Khan et al 2 pointed out that some reasons for performing quality assessment include: to determine a minimum quality threshold for the selection of the primary studies for a systematic review; to explore differences in quality as an explanation for heterogeneity in study results; to weigh the results in proportion to the quality in metaanalysis; and, more importantly, to guide interpretation of findings, help determine the strength of inferences, and guide recommendations for future research and clinical practice.The assessment of the quality of controlled trials is essential because variations in the quality of trials can affect the conclusions about the existing evidence. 3 In a review of trials evaluating primarily medical treatments, Moher and colleagues 4,5 demonstrated that trials that did not include features such as blinding and allocation concealment tended to report an exaggerated treatment effect compared with trials that did include these features. These facts emphasize the importance of methodological quality assessment in order to provide accurate information on therapeutic effects.Trial quality can be divided ...

show abstract

“…In 1994, in an attempt to address the ‘wide chasm’ between what a randomized controlled trial (RCT) should report and what is actually reported, the Standards of Reporting Trials Group developed a ‘proposal for the structured reporting of RCT’s’ [8]. This proposal later became the CONSORT statement (CONsolidated Standards Of Reporting Trials), which is one of the earliest and most well-established reporting guidelines [9].…”

Section: Introductionmentioning

confidence: 99%

Development and validation of the guideline for reporting evidence-based practice educational interventions and teaching (GREET)

et al. 2016

View full text Add to dashboard Cite

BackgroundThe majority of reporting guidelines assist researchers to report consistent information concerning study design, however, they contain limited information for describing study interventions. Using a three-stage development process, the Guideline for Reporting Evidence-based practice Educational interventions and Teaching (GREET) checklist and accompanying explanatory paper were developed to provide guidance for the reporting of educational interventions for evidence-based practice (EBP). The aim of this study was to complete the final development for the GREET checklist, incorporating psychometric testing to determine inter-rater reliability and criterion validity.MethodsThe final development for the GREET checklist incorporated the results of a prior systematic review and Delphi survey. Thirty-nine items, including all items from the prior systematic review, were proposed for inclusion in the GREET checklist. These 39 items were considered over a series of consensus discussions to determine the inclusion of items in the GREET checklist. The GREET checklist and explanatory paper were then developed and underwent psychometric testing with tertiary health professional students who evaluated the completeness of the reporting in a published study using the GREET checklist. For each GREET checklist item, consistency (%) of agreement both between participants and the consensus criterion reference measure were calculated. Criterion validity and inter-rater reliability were analysed using intra-class correlation coefficients (ICC).ResultsThree consensus discussions were undertaken, with 14 items identified for inclusion in the GREET checklist. Following further expert review by the Delphi panelists, three items were added and minor wording changes were completed, resulting in 17 checklist items. Psychometric testing for the updated GREET checklist was completed by 31 participants (n = 11 undergraduate, n = 20 postgraduate). The consistency of agreement between the participant ratings for completeness of reporting with the consensus criterion ratings ranged from 19 % for item 4 Steps of EBP, to 94 % for item 16 Planned delivery. The overall consistency of agreement, for criterion validity (ICC 0.73) and inter-rater reliability (ICC 0.96), was good to almost perfect.ConclusionThe final GREET checklist comprises 17 items which are recommended for reporting EBP educational interventions. Further validation of the GREET checklist with experts in EBP research and education is recommended.Electronic supplementary materialThe online version of this article (doi:10.1186/s12909-016-0759-1) contains supplementary material, which is available to authorized users.

show abstract

A Proposal for Structured Reporting of Randomized Controlled Trials

Cited by 239 publications

References 52 publications

Evaluating Behavioral Health Interventions for Military-Connected Youth: A Systematic Review

Evaluating Behavioral Health Interventions for Military-Connected Youth: A Systematic Review

Scales to Assess the Quality of Randomized Controlled Trials: A Systematic Review

Development and validation of the guideline for reporting evidence-based practice educational interventions and teaching (GREET)

Contact Info

Product

Resources

About