Methodological Deficits in Diagnostic Research Using ‘-Omics’ Technologies: Evaluation of the QUADOMICS Tool and Quality of Recently Published Studies

Abstract

BackgroundQUADOMICS is an adaptation of QUADAS (a quality assessment tool for use in systematic reviews of diagnostic accuracy studies), which takes into account the particular challenges presented by ‘-omics’ based technologies. Our primary objective was to evaluate the applicability and consistency of QUADOMICS. Subsequently we evaluated and describe the methodological quality of a sample of recently published studies using the tool.Methodology/Principal Findings45‘-omics’- based diagnostic studies were identified by systematic search of Pubmed using suitable MeSH terms (“Genomics”, “Sensitivity and specificity”, “Diagnosis”). Three investigators independently assessed the quality of the articles using QUADOMICS and met to compare observations and generate a consensus. Consistency and applicability was assessed by comparing each reviewer's original rating with the consensus. Methodological quality was described using the consensus rating. Agreement was above 80% for all three reviewers. Four items presented difficulties with application, mostly due to the lack of a clearly defined gold standard. Methodological quality of our sample was poor; studies met roughly half of the applied criteria (mean ± sd, 54.7±18.4%). Few studies were carried out in a population that mirrored the clinical situation in which the test would be used in practice, (6, 13.3%); none described patient recruitment sufficiently; and less than half described clinical and physiological factors that might influence the biomarker profile (20, 44.4%).ConclusionsThe QUADOMICS tool can consistently be applied to diagnostic ‘-omics’ studies presently published in biomedical journals. A substantial proportion of reports in this research field fail to address design issues that are fundamental to make inferences relevant for patient care.

Clinical validity of detecting K-ras mutations for the diagnosis of exocrine pancreatic cancer: a prospective study in a clinically-relevant spectrum of patients

“…In contingency tables, Fisher's exact test for homogeneity or independence was applied to assess the relationship between two categorical variables. Diagnostic accuracy of detecting K-ras mutation was expressed in terms of sensitivity and specificity with their corresponding 95% confidence intervals (CI) [26,28]. Data analysis was performed with SPSS 15.0 and Stata 8.0.…”

“…The diagnostic validity of a potential test should be evaluated in relevant populations [26][27][28][29][30]. It is particularly important that the ultimate stage of validation be carried out in a population as close as possible to that in which the test would be used in practice.…”

MECHANISMS IN ENDOCRINOLOGY: Proteomic biomarkers of type 2 diabetes mellitus risk in women with polycystic ovary syndrome

“…The methodological quality of primary studies applying proteomics in T2DM was determined using the QUADOMICS Tool, an adaptation of a quality assessment tool for use in systematic reviews of the diagnostic accuracy studies (QUADAS), which takes into account the particular challenges encountered in '-omics'-based techniques (17). The criteria that comprise the QUADOMICS Tool are shown in Fig.…”

“…Assessing the quality of the relevant studies Table 2 summarizes the quality assessment process in accordance with the QUADOMICS Tool (17). Six out of the nine studies were classified as HQ fulfilling at least ten out of the 16 criteria (30,31,33,34,35,37).…”

“…The three remaining studies were classified as LQ achieving 8 or 9 out of the 16 quality (32,36,38). Of particular note was that none of the studies explicitly stated that the index test results were interpreted without knowledge of the results of the reference standard and vice versa, thus failing to achieve the 12th and 13th criteria of the QUADOMICS (17). Furthermore, the 2nd and 14th quality criteria of the QUADOMICS (Fig.…”