Detecting fabrication in large-scale molecular omics data

Bradshaw, Michael S.; Payne, Samuel H.

doi:10.1371/journal.pone.0260395

Cited by 7 publications

(7 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Targeted gene research may represent an attractive topic for paper mills because the associated experimental results are easy to fabricate (Figure 2 ). In contrast to the fabrication of genome-wide research that has been estimated to require similar effort as the acquisition of genuine data ( 74 ), targeted gene research is easier to fabricate than to produce through genuine effort ( 27 , 38 ) (Figure 2 ). Targeted gene research uses experimental techniques that generate small individual datasets that are easy to invent, with access to only basic text and image processing software.…”

Section: Why Human Gene Research Could Be Particularly Vulnerable To ...mentioning

confidence: 99%

Protection of the human gene research literature from contract cheating organizations known as research paper mills

Byrne

Park

Richardson

et al. 2022

Nucleic Acids Research

View full text Add to dashboard Cite

Human gene research generates new biology insights with translational potential, yet few studies have considered the health of the human gene literature. The accessibility of human genes for targeted research, combined with unreasonable publication pressures and recent developments in scholarly publishing, may have created a market for low-quality or fraudulent human gene research articles, including articles produced by contract cheating organizations known as paper mills. This review summarises the evidence that paper mills contribute to the human gene research literature at scale and outlines why targeted gene research may be particularly vulnerable to systematic research fraud. To raise awareness of targeted gene research from paper mills, we highlight features of problematic manuscripts and publications that can be detected by gene researchers and/or journal staff. As improved awareness and detection could drive the further evolution of paper mill-supported publications, we also propose changes to academic publishing to more effectively deter and correct problematic publications at scale. In summary, the threat of paper mill-supported gene research highlights the need for all researchers to approach the literature with a more critical mindset, and demand publications that are underpinned by plausible research justifications, rigorous experiments and fully transparent reporting.

show abstract

Section: Why Human Gene Research Could Be Particularly Vulnerable To ...mentioning

confidence: 99%

Protection of the human gene research literature from contract cheating organizations known as research paper mills

Byrne

Park

Richardson

et al. 2022

Nucleic Acids Research

View full text Add to dashboard Cite

show abstract

“…Unlike images, data fabrication in numerical tables can be subtle and hard to detect. In 2020, Bradshaw and Payne examined methods to detect fabricated numerical omics data, 126 concluding that methods originally developed to detect fraud in banking and insurance would be applicable to scientific data.…”

Section: Research Integritymentioning

confidence: 99%

“…Manipulated or fabricated data can be detected by a variety of algorithms leveraging number theory metrics unrelated to study design, such as Benford’s Law 130 used by Bradshaw and Payne, 126 and Zipf’s Law. 131 The workshop demonstrated that some existing models for predicting mass spectra violate some of these laws.…”

Section: Research Integritymentioning

confidence: 99%

Toward an Integrated Machine Learning Model of a Proteomics Experiment

et al. 2023

Self Cite

View full text Add to dashboard Cite

In recent years machine learning has made extensive progress in modeling many aspects of mass spectrometry data. We brought together proteomics data generators, repository managers, and machine learning experts in a workshop with the goals to evaluate and explore machine learning applications for realistic modeling of data from multidimensional mass spectrometry-based proteomics analysis of any sample or organism. Following this sample-to-data roadmap helped identify knowledge gaps and define needs. Being able to generate bespoke and realistic synthetic data has legitimate and important uses in system suitability, method development, and algorithm benchmarking, while also posing critical ethical questions. The interdisciplinary nature of the workshop informed discussions of what is currently possible and future opportunities and challenges. In the following perspective we summarize these discussions in the hope of conveying our excitement about the potential of machine learning in proteomics and to inspire future research.

show abstract

“…We used t-statistics to test for issues in baseline tables, but other methods could be applied such as Benford's law [22,43].…”

Section: Relation Previous Workmentioning

confidence: 99%

Automated detection of over- and under-dispersion in baseline tables in randomised controlled trials

Barnett

2022

Preprint

View full text Add to dashboard Cite

Background: Papers describing the results of a randomised trial should include a table of summary statistics that compares randomised groups at baseline. Researchers who fraudulently generate trials often create randomised groups that are implausibly similar (under-dispersed) or accidentally create large differences between groups (over-dispersed) because they do not understand how to create realistic summary statistics from truly random data. We aimed to automatically screen for under- and over-dispersion in the baseline tables of published randomised trials.Methods: We examined randomised controlled trials published in open access health and medical journals on PubMed Central. We estimated probability that a trial’s summary statistics were under- or over-dispersed using a Bayesian model that examined the distribution of t-statistics for the between-group differences from the baseline table, and compared this with an expected distribution without dispersion. We used a simulation study to test the ability of the model to find over- or under-dispersion and compared its performance with an existing test of dispersion based on a uniform test of p-values.Results: The algorithm had a relatively good accuracy for extracting the data from baseline tables, matching well on the size of the tables and sample size. Using t-statistics in the Bayesian model out-performed the uniform test of p-values, as it had a higher probability of detecting over-dispersed data and lower false positive percentage for skewed data that was not under- or over-dispersed. For the trials published on PubMed Central, some tables were flagged because: they were not from trials, had a non-standard presentation, or had likely reporting errors. The algorithm also flagged some trials as under-dispersed where there was a striking similarity between groups.Conclusion: Due to the variance in reporting across journals, the current method may not be suitable for automated screening for fraud of all submitted trials, but could be useful in targeted checks of suspected trials. If used in automated screening of papers at the submission stage, it would likely flag many instances of non-standard table presentation and poor reporting.

show abstract

Detecting fabrication in large-scale molecular omics data

Cited by 7 publications

References 30 publications

Protection of the human gene research literature from contract cheating organizations known as research paper mills

Protection of the human gene research literature from contract cheating organizations known as research paper mills

Toward an Integrated Machine Learning Model of a Proteomics Experiment

Automated detection of over- and under-dispersion in baseline tables in randomised controlled trials

Contact Info

Product

Resources

About