Methods to assess research misconduct in health-related research: A scoping review

Li, Wentao; Eekelen, Rik van; Wang, Rui; Showell, Marian; Mol, Ben W.; Wely, Madelon van

doi:10.1016/j.jclinepi.2021.05.012

Cited by 43 publications

(43 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As incorrect nucleotide sequences are unlikely to be found in all problematic gene research articles, future research should also combine analyses of nucleotide sequences with other features of concern such as manipulated or recurring experimental images and data ( 24 , 67 , 68 Preprint ). Furthermore, as many research fields do not use verifiable reagents, other methodologies are required to assess publications that describe epidemiological studies or clinical trials ( 69 , 70 ).…”

Section: Discussionmentioning

confidence: 99%

Identification of human gene research articles with wrongly identified nucleotide sequences

et al. 2022

View full text Add to dashboard Cite

Nucleotide sequence reagents underpin molecular techniques that have been applied across hundreds of thousands of publications. We have previously reported wrongly identified nucleotide sequence reagents in human research publications and described a semi-automated screening tool Seek & Blastn to fact-check their claimed status. We applied Seek & Blastn to screen >11,700 publications across five literature corpora, including all original publications in Gene from 2007 to 2018 and all original open-access publications in Oncology Reports from 2014 to 2018. After manually checking Seek & Blastn outputs for >3,400 human research articles, we identified 712 articles across 78 journals that described at least one wrongly identified nucleotide sequence. Verifying the claimed identities of >13,700 sequences highlighted 1,535 wrongly identified sequences, most of which were claimed targeting reagents for the analysis of 365 human protein-coding genes and 120 non-coding RNAs. The 712 problematic articles have received >17,000 citations, including citations by human clinical trials. Given our estimate that approximately one-quarter of problematic articles may misinform the future development of human therapies, urgent measures are required to address unreliable gene research articles.

show abstract

Section: Discussionmentioning

confidence: 99%

Identification of human gene research articles with wrongly identified nucleotide sequences

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Evidence users may subsequently investigate such trials further or systematic reviewers may consider sensitivity analyses excluding such trials from meta-analyses. We direct readers to other sources that describe these methods (44).…”

Section: Discussionmentioning

confidence: 99%

“…Box 3: Methods to assess for fabrication and falsification in clinical trials (44) 1. Judge whether the reported recruitment speed is feasible given local disease patterns, trial eligibility criteria, and capacity of the recruiting centres (45, 46).…”

Section: Discussionmentioning

confidence: 99%

The trustworthiness and impact of trial preprints for COVID-19 decision-making: A methodological study

Zeraatkar

Pitre

Leung

et al. 2022

Preprint

View full text Add to dashboard Cite

Purpose: To assess the trustworthiness and impact of preprint trial reports during the COVID-19 pandemic. Data sources: WHO COVID-19 database and the L-OVE COVID-19 platform by the Epistemonikos Foundation (up to August 3rd, 2021) Design: We compare the characteristics of COVID-19 trials with and without preprints, estimate time to publication of COVID-19 preprint reports, describe discrepancies in key methods and results between preprint and published trial reports, report the number of retracted preprints and publications, and assess whether including versus excluding preprint reports affects meta-analytic estimates and the certainty of evidence. For the effects of eight therapies on mortality and mechanical ventilation, we performed meta-analyses including preprints and excluding preprints at 1 month, 3 months, and 6 months after the first trial addressing the therapy became available either as a preprint or publication (120 meta-analyses in total). Results: We included 356 trials, 101 of which are only available as preprints, 181 as journal publications, and 74 as preprints first and subsequently published in journals. Half of all preprints remain unpublished at six months and a third at one year. There were few important differences in key methods and results between trial preprints and their subsequent published reports. We identified four retracted trials, three of which were published in peer-reviewed journals. With two exceptions (2/60; 3.3%), point estimates were consistent between meta-analyses including versus excluding preprints as to whether they indicated benefit, no appreciable effect, or harm. There were nine comparisons (9/60; 15%) for which the rating of the certainty of evidence differed when preprints were included versus excluded, for four of these comparisons the certainty of evidence including preprints was higher and for five of these comparisons the certainty of evidence including preprints was lower. Limitations: The generalizability of our results is limited to COVID-19. Preprints that are subsequently published in journals may be the most rigorous and may not represent all trial preprints. Conclusion: We found no compelling evidence that preprints provide less trustworthy results than published papers. We show that preprints remain the only source of findings of many trials for several months, for a length of time that is unacceptable in a health emergency. We show that including preprints may affect the results of meta-analyses and the certainty of evidence. We encourage evidence users to consider data from preprints in contexts in which decisions are being made rapidly and evidence is being produced faster than can be peer-reviewed.

show abstract

“…Manually extracting data from papers is time consuming and automatic data extraction would be a useful advance. Automated detection tools would save time and increase scrutiny [11].…”

Section: Introductionmentioning

confidence: 99%

“…These include planned or unplanned factors to do with randomisation, such as dynamic randomisation to create highly comparable groups, or subversion of the random allocation [16]. Differences could also arise at the reporting stage, such as mislabelling summary statistics or reporting errors [11]. Our aim is to create a procedure that could be used to ag potential problems at the submission stage, which would help researchers improve their paper prior to publication [17].…”

Section: Introductionmentioning

confidence: 99%

Automated detection of over- and under-dispersion in baseline tables in randomised controlled trials

Barnett

2022

Preprint

View full text Add to dashboard Cite

Background: Papers describing the results of a randomised trial should include a table of summary statistics that compares randomised groups at baseline. Researchers who fraudulently generate trials often create randomised groups that are implausibly similar (under-dispersed) or accidentally create large differences between groups (over-dispersed) because they do not understand how to create realistic summary statistics from truly random data. We aimed to automatically screen for under- and over-dispersion in the baseline tables of published randomised trials.Methods: We examined randomised controlled trials published in open access health and medical journals on PubMed Central. We estimated probability that a trial’s summary statistics were under- or over-dispersed using a Bayesian model that examined the distribution of t-statistics for the between-group differences from the baseline table, and compared this with an expected distribution without dispersion. We used a simulation study to test the ability of the model to find over- or under-dispersion and compared its performance with an existing test of dispersion based on a uniform test of p-values.Results: The algorithm had a relatively good accuracy for extracting the data from baseline tables, matching well on the size of the tables and sample size. Using t-statistics in the Bayesian model out-performed the uniform test of p-values, as it had a higher probability of detecting over-dispersed data and lower false positive percentage for skewed data that was not under- or over-dispersed. For the trials published on PubMed Central, some tables were flagged because: they were not from trials, had a non-standard presentation, or had likely reporting errors. The algorithm also flagged some trials as under-dispersed where there was a striking similarity between groups.Conclusion: Due to the variance in reporting across journals, the current method may not be suitable for automated screening for fraud of all submitted trials, but could be useful in targeted checks of suspected trials. If used in automated screening of papers at the submission stage, it would likely flag many instances of non-standard table presentation and poor reporting.

show abstract

Methods to assess research misconduct in health-related research: A scoping review

Cited by 43 publications

References 56 publications

Identification of human gene research articles with wrongly identified nucleotide sequences

Identification of human gene research articles with wrongly identified nucleotide sequences

The trustworthiness and impact of trial preprints for COVID-19 decision-making: A methodological study

Automated detection of over- and under-dispersion in baseline tables in randomised controlled trials

Contact Info

Product

Resources

About